From 85fcac2a61b049ae29a081843e5121739c7378ab Mon Sep 17 00:00:00 2001 From: Kovid Goyal Date: Sun, 14 Jan 2024 21:27:55 +0530 Subject: [PATCH] Add throughput performance numbers --- docs/performance.rst | 122 +++++++++++++++++++++++++++++++++---------- 1 file changed, 93 insertions(+), 29 deletions(-) diff --git a/docs/performance.rst b/docs/performance.rst index c68de4c26..6fe582537 100644 --- a/docs/performance.rst +++ b/docs/performance.rst @@ -3,10 +3,14 @@ Performance The main goals for |kitty| performance are user perceived latency while typing and "smoothness" while scrolling as well as CPU usage. |kitty| tries hard to -find an optimum balance for these. To that end it keeps a cache of each rendered -glyph in video RAM so that font rendering is not a bottleneck. Interaction with -child programs takes place in a separate thread from rendering, to improve -smoothness. +find an optimum balance for these. To that end it keeps a cache of each +rendered glyph in video RAM so that font rendering is not a bottleneck. +Interaction with child programs takes place in a separate thread from +rendering, to improve smoothness. Parsing of the byte stream is done using +`vector CPU instructions +`__ for +maximum performance. Updates to the screen typically require sending just a few +bytes to the GPU. There are two config options you can tune to adjust the performance, :opt:`repaint_delay` and :opt:`input_delay`. These control the artificial delays @@ -15,19 +19,84 @@ introduced into the render loop to reduce CPU usage. See option to further decrease latency at the cost of some `screen tearing `__ while scrolling. -You can generate detailed per-function performance data using -`gperftools `__. Build |kitty| with -``make profile``. Run kitty and perform the task you want to analyse, for -example, scrolling a large file with :program:`less`. After you quit, function -call statistics will be printed to STDOUT and you can use tools like -*KCachegrind* for more detailed analysis. +Benchmarks +------------- -Here are some CPU usage numbers for the task of scrolling a file continuously in -:program:`less`. The CPU usage is for the terminal process and X together and is -measured using :program:`htop`. The measurements are taken at the same font and -window size for all terminals on a ``Intel(R) Core(TM) i7-4820K CPU @ 3.70GHz`` -CPU with a ``Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde XT [Radeon HD -7770/8760 / R7 250X]`` GPU. +Measuring terminal emulator performance is fairly subtle, there are three main +axes on which performance is measured: Energy usage for typical tasks, +Keyboard to screen latency, and throughput (processing large amounts of data). + +Keyboard to screen latency +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This is measured either with dedicated hardware, or software such as `Typometer +`__. Third party measurements comparing +kitty with other terminal emulators on various systems show kitty has best in +class keyboard to screen latency. + +`Hardware based measurement on macOS +`__ show that kitty and +Apple's Terminal.app share the crown for best latency. These +measurements were done with :opt:`input_delay` at its default value of ``3 ms`` +which means kitty's actual numbers would be even lower. + +`Typometer based measurements on Linux +`__ +show that kitty has far and away the best latency of the terminals tested. + +Throughput +^^^^^^^^^^^^^^^^ + +kitty has a builtin kitten to measure throughput, it works by dumping large +amounts of data of different types into the tty device and measuring how fast +the terminal parses and responds to it. The measurements below were taken with +the same font, font size and window size for all terminals, and default +settings, on the same computer. They clearly show kitty has the fastest +throughput. To run the tests yourself, run ``kitten __benchmark__`` in the +terminal emulator you want to test, where the kitten binary is part of the +kitty install. + +The numbers are megabytes per second of data that the terminal +processes. Measurements were taken under Linux/X11 with a ``AMD Ryzen 7 PRO +5850U``. Entries are in order of decreasing performance. kitty is clearly +faster than the rest. + +================ ====== ======= ===== ====== ======= +Terminal ASCII Unicode CSI Images Average +================ ====== ======= ===== ====== ======= +kitty 0.33 88.3 74.1 41.2 235.8 109.85 +alacritty 0.13.1 43.1 46.5 32.5 94.1 54.05 +wezterm 20230712 16.4 26.0 11.1 140.5 48.5 +xterm 389 47.7 18.3 0.6 56.3 30.72 +konsole 23.08.04 25.2 37.7 23.6 23.4 27.48 +================ ====== ======= ===== ====== ======= + + +.. note:: + + By default, the benchmark kitten suppress actual rendering, to better focus + on parser speed, you can pass it the ``--render`` flag to not suppress + rendering. However, modern terminals typically render asynchronously, + therefore the numbers are not really useful for comparison. However, even + with rendering enabled kitty is still much faster than all the rest. For + brevity those numbers are not included. + +.. note:: + + gnome-terminal is left out as I could not get it to start on my system + and foot is left out as it does not run under X11. + + +Energy usage +^^^^^^^^^^^^^^^^^ + +Sadly, I do not have the infrastructure to measure actual energy usage so CPU +usage will have to stand in for it. Here are some CPU usage numbers for the +task of scrolling a file continuously in :program:`less`. The CPU usage is for +the terminal process and X together and is measured using :program:`htop`. The +measurements are taken at the same font and window size for all terminals on a +``Intel(R) Core(TM) i7-4820K CPU @ 3.70GHz`` CPU with a ``Advanced Micro +Devices, Inc. [AMD/ATI] Cape Verde XT [Radeon HD 7770/8760 / R7 250X]`` GPU. ============== ========================= Terminal CPU usage (X + terminal) @@ -40,21 +109,16 @@ gnome-terminal 15 - 17% konsole 29 - 31% ============== ========================= - As you can see, |kitty| uses much less CPU than all terminals, except xterm, but its scrolling "smoothness" is much better than that of xterm (at least to my, admittedly biased, eyes). +Instrumenting kitty +----------------------- -.. _perf-cat: - -.. note:: - - Some people have asked why kitty does not perform better than terminal XXX - in the test of sinking large amounts of data, such as catting a large text - file. The answer is because this is not a goal for kitty. kitty deliberately - throttles input parsing and output rendering to minimize resource usage - while still being able to sink output faster than any real world program can - produce it. Reducing CPU usage, and hence battery drain while achieving - instant response times and smooth scrolling to a human eye is a far more - important goal. +You can generate detailed per-function performance data using +`gperftools `__. Build |kitty| with +``make profile``. Run kitty and perform the task you want to analyse, for +example, scrolling a large file with :program:`less`. After you quit, function +call statistics will be printed to STDOUT and you can use tools like +*KCachegrind* for more detailed analysis.