fixed our website crash during running the automation testing because of my local environments couldn’t have a proper resources (in this case is low memory), i don’t expect that it would be piqued my interest when trying to unpack the rootcause and delve deeper into something “unknown” for me
so basically, when the website crash during running the automation in my local, it’s a multi process OS-level application. perhaps i can simplified like this
- automation start running using playwright
- 1 main browser process (here i’m using chromium)
- it has a tab and frame
- GPU start process
- network also start the process (used for pass the data between process)
- each of these will interacts with: linux namespace, shared memory, GPU drivers and sandbox permissions
the crash occurs if one of those process fail to start → renderer stop → page crashed
and it’s particular interesting because the crash only happened during launch the website homepage, and turns out navigation at the initial stage can be considered the heaviest load during browser lifecycle. The browser must cover and process:
- spawn the renderer utilities
- start to initialize the sandbox → which has isolated the PID namespaces, user namespaces and seccomp filters so it can be authenticated the linux permission (https://www.kernel.org/doc/html/v4.19/userspace-api/seccomp_filter.html)
- once sandbox success, start allocate shm buffers
- GPU start rendering final UI layers by offloading task from CPU
- running the task to execute the javascript program (in this case is our platform)
- load everything (framework, media, fonts, assets, and so on)
after digging and deeper and check the trace log, one of the main reasons why it crashed on my local because it’s only has a fewer size of shm and the availability at that time is only 1 % out of my current tmpfs (70-90 MB out of 7.7 GB)
so, when the shm start filled up, it will be:
- kernel kills the process
- rendered utilities cant allocate any memory
- stop the renderer frames function alongside hold the javascript execution buffers
think of it like: automation running → browser process → renderer process → OS kernel → Hardware (CPU, GPU, shm)