Neovim: Fix Random Diagnostic Order

Hey there, fellow Neovim enthusiasts! Ever noticed how sometimes your diagnostics, especially those with the same severity level, seem to pop up in a rather haphazard order? It can be a bit jarring, can't it? You expect a certain flow, a predictable sequence, and then BAM! Randomness strikes. Well, you're not alone in this, and today, we're diving deep into why this happens and, more importantly, how we can get Neovim's diagnostics to play nicely and appear in a stable, predictable sequence. We'll explore the root cause and walk through the steps to ensure your development environment is as organized as your code should be.

Understanding Diagnostic Ordering in Neovim

Let's talk about diagnostics in Neovim. These are those helpful little markers your editor provides to flag potential issues in your code, like errors, warnings, or hints. Typically, Neovim is pretty smart about how it displays these. You can configure it to sort diagnostics by severity, meaning errors will show up before warnings, and warnings before hints. This is incredibly useful for prioritizing what needs your attention first. However, the issue arises when you have multiple diagnostics of the exact same severity level. For instance, if you have three warnings, Neovim needs a way to decide which of those three warnings should appear first, second, and third.

Ideally, we'd want a stable sort. What does that mean? It means that if two items are considered equal (in this case, having the same severity), their relative order from the original list should be preserved. Think of it like lining up people by height. If two people are the exact same height, a stable sort would keep them in the order they were originally standing. An unstable sort, on the other hand, might swap them around for no apparent reason. The problem reported is that Neovim's diagnostic sorting, when dealing with items of equal severity, appears to be unstable. This leads to the diagnostics being displayed in a random or unpredictable order each time you load or refresh them.

This lack of stability can be frustrating for a few reasons. Firstly, it breaks the expected predictability of your development environment. You might become accustomed to seeing certain types of warnings or hints in a particular order, and when that order changes, it can be a minor, but persistent, annoyance. Secondly, it can sometimes make it harder to track down the source of a particular diagnostic if its position shifts. While the message and location are the primary identifiers, a consistent order can provide an additional layer of mental scaffolding.

Why Does This Happen? The Lua Sort Nuance

The core of the problem, as identified in the report, lies within the underlying sorting mechanism. When Neovim's diagnostic system sorts diagnostics of the same severity, it uses Lua's built-in table.sort function. The crucial detail here is that, by default, Lua's table.sort is not guaranteed to be stable. This means that for elements that compare as equal, their original relative order might not be maintained. The implementation details of table.sort can vary, and while some underlying sort algorithms are inherently stable (like merge sort), others might not be (like quicksort, which is often used and can be unstable). This non-deterministic behavior for equal elements is the culprit behind the random ordering of diagnostics with the same severity.

Consider the example provided in the report: a list of diagnostics, all with severity = 2. When vim.diagnostic.config({ severity_sort = true, ... }) is active, Neovim attempts to sort these. Because they all have the same severity, the sort function looks at them as