Here's a step-by-step guide to visualize Linux system calls in real-time on Ubuntu using the perf tool:

Prerequisites

• Make sure you have the linux-tools-common and linux-tools-generic packages installed. You can install them using:

sudo apt install linux-tools-common linux-tools-generic

Visualizing System Calls in Real-Time

1. Open a terminal on your Ubuntu system.

2. Run the following command to start the perf trace and visualize system calls in real-time:
   

sudo perf trace --event 'sys_enter*,sys_exit*' --event 'sched:sched_switch' --event 'sched:sched_wakeup' --event 'sched:sched_process_exit' --event 'sched:sched_process_free' --event 'sched:sched_process_wait' --event 'sched:sched_stat_sleep' --event 'sched:sched_stat_wait' --event 'sched:sched_stat_yield' --event 'sched:sched_stat_runtime' --event 'sched:sched_process_hang' --event 'sched:sched_process_exec' --event 'sched:sched_migrate_task' --event 'sched:sched_process_fork' --event 'sched:sched_process_exit' --event 'sched:sched_process_free' --event 'sched:sched_process_wait' --event 'sched:sched_stat_sleep' --event 'sched:sched_stat_wait' --event 'sched:sched_stat_yield' --event 'sched:sched_stat_runtime' --event 'sched:sched_process_hang' --event 'sched:sched_process_exec' --event 'sched:sched_migrate_task' --event 'sched:sched_process_fork'

This command enables tracing for various system call events and scheduler events. It provides a comprehensive view of the system's behavior.

1. The trace will start immediately, and you'll see system call events scrolling in the terminal.

2. To stop the trace, press Ctrl+C.

3. After stopping the trace, perf will generate a report file named perf.data in the current directory.

4. To visualize the trace, run the following command:
   

sudo perf script | ./path/to/FlameGraph/stackcollapse-perf.pl | ./path/to/FlameGraph/flamegraph.pl > perf_viz.svg

Replace ./path/to/FlameGraph/stackcollapse-perf.pl and ./path/to/FlameGraph/flamegraph.pl with the actual paths to the stackcollapse-perf.pl and flamegraph.pl scripts from the FlameGraph repository.

1. This command generates an SVG file named perf_viz.svg that visualizes the system call trace as a flame graph.

2. Open the generated perf_viz.svg file in a web browser to view the flame graph.

The flame graph provides a visual representation of the system calls, their duration, and the call stack. The width of each bar represents the time spent in each function or system call, making it easier to identify performance bottlenecks and hotspots.

By following these steps, you can effectively visualize Linux system calls in real-time using perf and the FlameGraph tool on your Ubuntu system. This technique helps in understanding system behavior, identifying performance issues, and optimizing applications.

Citations:
[1] https://xitoring.com/kb/strace-monitor-linux-system-calls/
[2] https://opensource.com/article/19/10/strace
[3] https://manpages.ubuntu.com/manpages/trusty/man1/strace.1.html
[4] https://dev.to/narasimha1997/lets-build-a-simple-system-calls-monitoring-dashboard-using-bpftrace-and-streamlit-2m1n
[5] https://www.pingcap.com/blog/how-to-trace-linux-system-calls-in-production-with-minimal-impact-on-performance/
[6] https://www.howtogeek.com/732736/how-to-use-strace-to-monitor-linux-system-calls/
[7] https://en.wikipedia.org/wiki/Kernel_%28operating_system%29
[8] https://sysprog21.github.io/lkmpg/