BITS&CHIPS EVENT 2024

This talk outlines the practical application and assessment of large language models tailored to the code generation task. It will cover our approach for evaluating these models’ performance in real-world software development environments and how we analyzed the failed cases. I will also present promising research areas to address the existing challenges and improving these models.

At VBTI, we have worked on many projects integrating DL models into (robotic) applications, mostly in the agricultural field. Most of these projects have in common that the DL models need to run remotely in fields, have strict timing requirements and have a limited resource budget for inference. While most deep-learning engineers focus on training and evaluating models on workstation-like hardware, this is not enough for the practical applications we’re working on. A lot of work is needed to deploy trained models on resource-constrained embedded hardware. In this talk, I will walk you through our process of taking a trained model and deploying it on a device in the field, with taking special care to making the model run as fast as possible using TensorRT.

It is all about data. By now everyone knows data is valuable. We use analytics to understand and even make predictions about our world. But at the core of all these analytics is data.

The amount of data we collect has a direct impact on the accuracy of our predictions, but data collection can be a cumbersome and time-consuming process. Especially with large and complex systems, the effort is huge. Wouldn’t it be nice if we could start collecting this valuable data on a system’s behaviour even before the system itself was build?

Data simulation can be a valuable contribution to start testing early and ease the cost and effort of data collection and get right into the analytics. However, there are many challenges and limitations when it comes to statistical models and simulating real-world behavior. It’s about finding the right balance to interpret results. Sometimes you need to dive into the model’s parameters and sometimes you just need to chase that pigeon from the machine.

Manufacturers are often quite conservative in modernizing and upgrading the PLC systems that keep their factories running. This conservatism is due to the loss of system knowledge and insufficient test coverage, which make it challenging to modernize PLC hardware and make changes in software. In most cases, such changes introduce system failures and unwanted downtime. Did you know manufacturers are scouring the internet for decades-old PLC hardware to prevent factory downtime?

Building on the master project of Bas Beuting, Cordis, Axini and Capgemini have developed an integrated approach to reverse-engineer PLC software to Cordis Suite models, from which PLC code can be generated for a variety of PLC platforms, with functional preservation verified by model-based testing with Axini. In addition, the models secure the essential knowledge to maintain PLC software for the future. No more hoarding of PLC hardware, no more legacy PLC software! Interested to learn more? Join our talk!

Systems engineering complexity in the high-tech equipment industry has increased, as the systems themselves steadily evolved with increased functionality and complexity. Current systems have a long lifetime, undergo rapid technological advancement, are increasingly software-intense and connected to other systems. Growing system complexity, the global competition, successful adoption of new technologies as AI and scarcity of systems engineers provide challenges to continuous innovation of the high-tech equipment industry.

Improved systems architecting and systems engineering (SA/SE) methodologies and competence development are needed to address these challenges. TNO-ESI has created an outlook on the required SA/SE capabilities. This talk provides an overview of identified relevant trends and challenges across the high-tech equipment industry, the derived needs and a research outlook on what is needed to upgrade their systems architecting and systems engineering capabilities.

Tired of spending precious time managing bugs? What if half of them don’t even need fixing? At Choco, we faced this challenge by using AI to streamline our bug management process. From automating bug assignment with GPT to integrating Slack, Intercom and Jira with Zapier for seamless bug reporting, our AI-driven hackathons transformed how we handle bugs. Join this talk to discover how AI can not only automate but elevate your bug management, fostering efficiency and creativity in your workflows.

Canon Production Printing has always been a pioneer in the inkjet printing business. At CPP, we are proud that our quality excellence put our products in a class of their own. For this to happen, our state-of-the-art image quality control is one of the critical contributors enabling our customers to print job after job with wide range of mixed media and minimum waste, at photographic quality.

In our engines, each second, thousands of nozzles produce millions of very small drops resulting in thousands of printed sheets (or tens of square meters of print) per hour. When only one nozzle performs poorly, even for a second, the resulting printed images can be marked unacceptable in accordance with our high print quality standard. Therefore, a real-time detection of artifacts and initiation of the right follow-up action are crucial to ensure high quality for undisturbed long runs.

In this talk, we will share our ambition and challenges of developing and integrating models (based on machine learning and image processing techniques) for fast and accurate anomaly detection.

In this talk, we will explore the data analytics for Entune BuildingAI. This is a smart assistant to reduce energy consumption at large-scale facilities. We use greybox modeling, while keeping the customer in the loop with a GenAI assistant for feedback and interaction with your energy appliances.

As we dive into the data generated, we focus on what truly matters. How do we gauge performance effectively? What benchmarks ensure our customers’ satisfaction and comfort? These questions have driven us to develop our own set of key performance indicators, while keeping in line with industry standards. Join us to explore how our KPIs not only measure success but also provide actionable insights, ensuring energy efficiency and optimal performance.

Early 2024, a major migration of a vital part of the Philips Hue cloud back-end infrastructure was successfully completed. During this migration, one of world’s largest IoT fleets consisting of millions of devices was moved over without any disruption or consumers even noticing it in any shape or form. The main reason behind this success can be attributed to the use of the principles of site reliability engineering (SRE), which is what you get when you treat operations as if it is a software problem.

In this talk, it is explained what SRE is, what the implications are on your product and how it is perceived by the customer, your system architecture, on your engineering processes and even on some of your business processes. It also addresses the various lessons learned the hard way.

Legacy code is challenging to maintain and understand, yet it is crucial for the operation of complex systems, especially in the Dutch high-tech industry. Static analysis techniques are often used to extract insights from codebases. While reliable, these techniques require parser-specific and domain-specific knowledge, resulting in a steep learning curve that limits their adoption in industrial environment. In contrast, large language models (LLMs) excel in general human-machine interpretation and user-friendliness but lack the precise domain-specific knowledge needed for in-depth accurate code analysis.

At TNO-ESI, we are developing a hybrid method that combines traditional static analysis with LLMs to better support software understanding and development. By using parser-based static analysis, we extract code details into a graph database. We enable LLMs to interact with this graph database to provide natural language responses with accurate knowledge on the codebase. This hybrid approach demonstrates how LLM accuracy can be augmented using traditional static analysis methods and conversely traditional software engineering techniques can be enhanced by integrating abstraction capabilities and user-friendliness of LLMs, allowing users to obtain answers that neither static analysis tools nor LLMs could address independently.

Supervisory control is a key part of cyber-physical systems, to orchestrate all system resources to work together in a safe, correct and optimal way. Developing supervisory controllers by hand becomes increasingly challenging, for instance due to increasing performance demands, the need to support more and more system variants, shortage of skilled engineers, and so on. In contrast, with synthesis-based engineering (SBE), correct-by-construction supervisory controllers can automatically be computed based on requirement specifications of

The Robotics and Autonomous Systems (RAS) unit of the Royal Netherlands Army is shaping the future of military operations with its groundbreaking Sentinel project. This initiative aims to revolutionize battlefield tactics by integrating advanced unmanned autonomous systems into military scenarios. Sentinel focuses on designing, prototyping and demonstrating a comprehensive chain of functions, including human input, human-machine interfaces, threat and terrain analysis, doctrinal planning, robot control and autonomous execution. Collaborating with both military and civilian industries, the project leverages Agile and Scrum methodologies to create regular demonstrations. Currently, virtual showcases highlight the autonomous capabilities of unmanned ground vehicles (UGVs). Future phases will expand to include diverse operational environments and a variety of UxVs, enhancing their functions to further solidify the operational and tactical advantages of autonomous military robotics.

Did you know the healthcare sector contributes 4-6 percent to global greenhouse gas emissions, surpassing even the aviation industry? While AI holds the promise of reducing this footprint (for instance through optimized logistics and predictive maintenance), its own energy and water demands can also contribute to healthcare’s carbon emissions. Join this talk to discuss how to strike the right balance and find out how measuring the AI footprint can guide us toward more sustainable choices for healthcare. We will touch upon best practices for green AI and the innovation efforts at Philips to inspire a greener future.

Static code analysis and runtime tests in conjunction with code coverage are proven methods for improving code quality. This talk shows the advantages but also the limitations of both methods. To ensure good code quality, both processes must be used in a complementary manner. The talk also discusses suitable criteria for selecting tools for static code analysis and measuring code coverage.

With an increased dependence on digital systems and a rise in severe cyber security incidents and threats, it’s increasingly hard – if not impossible – for businesses to keep up in the rat race between attackers and defenders. While cyber security has transformed into an essential prerequisite for businesses to function, the products and systems we need to protect become increasingly complex, dynamic and exposed. A reactive approach to cyber security is no longer feasible, nor sufficient. Society requires cyber-resilient products and systems with the inherent ability to prevent, withstand and recover from incidents. Inherent cyber resilience of products and systems benefits business continuity and lowering the overall lifecycle costs by reducing damages, product updates and recalls. Yet, industry parties involved in systems design and engineering are having difficulty crafting a convincing business case.

In this talk, we will showcase TNO’s current research and vision on cyber-resilient system design, touching upon topics including design and lifecycle management, software verification and validation, and human-centric engineering. In addition, we will discuss and demonstrate the possibilities and limitations of TNO’s Self-Healing for Cyber Security (SH4CS) software. Inspired by biological mechanisms in cells and the human immune system, this novel software aims to make products and systems autonomously cope with any type of operational (runtime) abnormalities.

The design of (cryptographic) hardware components in a hardware design language (eg VHDL, Verilog) is a manual task and it is hard to prove that the design adheres to the functional and security requirements. Formal verification can be used to provide guarantees on the security of the mathematical design of cryptography or its software implementation. Some formal verification methods are already applied in the field of hardware design, but these are all manual. I will talk about our research into automatic formal verification of hardware designs and how these techniques can even automatically generate the VHDL/Verilog code based on the specification.

Computers are deterministic, but the real world is full of unpredictability. In critical decision-making, it is crucial for AI models to reflect this uncertainty. Imagine being part of a coast guard rescue operation. A deterministic AI system could wrongly distinguish between a drifting piece of debris and a person in distress without providing any additional information. If an AI system instead conveyed its uncertainty about the prediction, would you send someone out to check?

Probabilistic programming excels in such scenarios by merging statistical modeling with AI and machine learning, giving models the ability to handle uncertainty effectively. This talk provides an introduction to probabilistic programming, highlighting its crucial role in AI and machine learning for decision-making. It delves into the motivations behind adopting probabilistic programming and explores practical examples and key concepts, such as the Pyro framework, demonstrating the robust and reliable nature of AI systems powered by probabilistic programming.

The incredibly fast advances in hardware and software around machine learning enabled massively large and incredibly complex AI models to be extremely accurate for various tasks, from solving ‘simple’ image classification tasks to generating solutions for equations or coding assignments, and even acting as a sparring partner for various games.

However, all these advances do not come for free: the entire lifecycle of machine learning models requires massive computation, from architectural search to hyperparameter tuning and inference at scale.

This computation, in turn, consumes significant amounts of energy and, ultimately, has a non-negligible impact on the long-term sustainability of the field.

In this talk, we argue we need to define the metrics and design the tools required to assess the sustainability impact of AI models during their entire lifecycle. We present a couple of case studies and illustrate the use of these metrics to assess the sustainability cost associated with these models. Furthermore, by means of simple performance and efficiency models, we also illustrate how to monitor, label and possibly improve the sustainability of machine learning models. We conclude that sustainability models are powerful tools to raise awareness (and possibly alleviate concerns) about the sustainability impact of complex AI models, as long as the communities of AI developers and AI users are willing to adopt and/or endorse them.

Code coverage is a key metric for software quality assurance, mandated by safety standards such as IEC 61508 and ISO 26262. We focus on branch coverage and modified condition/decision coverage (MC/DC), revealing differences in the calculations of measurement tools and the varying results between tools. This talk highlights two key challenges in coverage measurement: calculation oversights and gaps. It also addresses the pitfalls of deriving test cases from source code in the pursuit of 100 percent coverage. Strategies and resources for achieving unattainable 100 percent coverage are explored, along with the lack of standardized nomenclature for coverage metrics, and misconceptions are dispelled. In addition, insights will be shared on how to authenticate the quality of test cases that contribute to improved software reliability. The presentation is designed for both beginners and experts, and includes interactive examples for a comprehensive understanding.

Modern systems such as large-scale distributed radio telescopes face strong requirements in data acquisition in the RF front-end and data transport to the back-end. We will present a use case based on a large-scale distributed radio telescope system. In the radio telescope front-end, the antenna signals are digitized, filtered and pre-processed. The resulting data is transmitted in a streaming manner over Ethernet to a centralized location where multiple antennas are computationally combined into science products.

RFSoC FPGAs provide ample high-speed interfaces. However, design of custom FPGA boards, especially with high-speed interfaces is a labor-intensive and expert task. iWave’s system-on-modules integrate the RFSoC FPGA on a module that can be used in custom designs.

Join us to find out how an RFSoC system-on-module can meet the system challenges of a large-scale distributed radio telescope.