Media CI improvements over time and the start of testing Sashiko as another CI report bot

1. Introduction

There have been lots of discuss and efforts over the last years with regards to improving the quality of media patches.

In the past, Mauro had my own set of scripts to check for regressions and do media compiling per patch while reviewing submitted patches. Later on, Hans came up with his set of scripts as well. Both approaches rely not only on kernel compilation but they also run two static analyzer tools: sparse and smatch.

Media ML also receives since a long time ago mails from bots like Syzbot, kernel test robot - and there were some other bots as well that sent us reviews in the past.

We also added the first CI using Jenkins, which we still use for userspace tools.

There was also Coverity which is a paid tool which usage is free for open source developers, which provides more sophisticated analysis of the Kernel code. Its last run was 8 months ago, so probably nobody is using it anymore.

Starting on Jan, 2024, Ricardo started writing the media-ci code which is now used to review all Linux Kernel media patches and merges.

There is a common pattern on all such tools: they produce false positives. Last week we just addressed one such issue on media-CI: if, for whatever reason, a CI job is interrupted (system error), it was reporting it the same way as if the patch had a high relevance issue.

Btw, media-ci also have on its tree a list of false-positive issues to be ignored, which needs to be reviewed and eventually updated on every new kernel release.

So, 2,5 years after the start of process, we're still fine-tuning media CI to reduce false-positives. There have been more than 300 merge requests for Media-CI since v0, and the rate if MR is not slowing down.

That's perfectly fine, as having bots doing their work properly requires hard work.

Ricardo and other media-ci contributors: thank you for your work there!

So, maybe despite common sense, all CI bots have issues and can generate false positives. In doubt, authors should use their own good sense, and, when in doubt, check with a more experienced developer to double-check CI findings.

2. Sashiko

Maybe to cover the space left by Coverity, there is now a new project aiming to do some static analysis of the kernel code using LLM. Version 0.1.0 is from March, 2026, so this is a brand new approach. As such, it can also generate false positives, and, more importantly, as it is a new tool, it will certainly need some time to become mature.

One of the strengths of using static analysis tools is that they are usually better to review error paths that may slip on normal reviews.

With LLM-based tools, the hardest part is to have a set of prompts that teach the tool what it should produce, what kind of reviews are important and what it can consider as false positives.

During the last Media Summit, we reached a consensus about start testing it. There was some doubts about who should be receiving Sashiko's return. So, there was an inquiry with Media Summit participants later on to double check how Sashiko tests would work on media. There was non consensus, so we opted to be conservative: Sashiko's e-mails will be sent to media-ci@linuxtv.org mailing list without c/c the patch author. Our patchwork instance is picking such e-mails and adding them as checks at https://patchwork.linuxtv.org.

Subscribing to it can be done via mailman3 Postorious interface:

https://linuxtv.org/mailman3/postorius/lists/media-ci.linuxtv.org/

(to avoid SPAM, only members can post to it without moderation)

Archives are available at:

https://linuxtv.org/mailman3/hyperkitty/list/media-ci@linuxtv.org/

and:

https://sashiko.dev/#/?list=org.kernel.vger.linux-media

Yesterday, Sashiko v 0.2.5 wast tagged containing IMHO two relevant set of changes:

1. Initial support for linux-media was added. This includes a new prompt meant to cover media specifics: https://github.com/sashiko-dev/sashiko/blob/main/third_party/prompts/kernel/subsystem/media.md These prompts were generated on feedback we gave to the Sashiko maintainers. See this issue:

   https://github.com/sashiko-dev/sashiko/issues/334

2. It should be possible now to use Ollama to run it locally using open-weight models. This may help media developers to test how changes at the media prompt may affect the tool. Still, running it locally takes maybe 10-20 minutes per patch, and most models are bad honoring JSON format, which is currently needed by the Ollama integration.

With that in place, we guess we can now start testing the tool.

For such purpose, we kindly ask all contributors (patch authors and reviewers) to subscribe to media-ci@linuxtv.org, check Sashiko output and in the event of finding false positives share with media-ci ML so we can improve the prompts or create a new issue at:

https://github.com/sashiko-dev/sashiko/issues

This way, Sashiko developers can work to improve linux-media specifics.

Btw, the same applies to media CI: if you find false positive issues there, feel free to open/update issues at:

https://gitlab.freedesktop.org/linux-media/media-ci/-/work_items

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Signed-off-by: Hans Verkuil <hverkuil@kernel.org>

Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>

mchehab