DCONSULTED
DConsulted • Engineering Insights
Optimize Performance, Lower Costs with Data-Oriented Design
Published September 26, 2024

Optimize Performance, Lower Costs with Data-Oriented Design

Optimize Performance, Lower Costs with Data-Oriented Design

Optimize Performance and Lower Costs with Data-Oriented Design

Facebook
Twitter
LinkedIn

There are two main impediments in automotive software development: high processor cost and poor performance. Performance is particularly important for any safety-critical system that relies on real-time responses, low latency, and high throughput. 

 

Adding more software on a single processor can reduce the hardware needed, keeping tasks on track and lowering costs. High-performance processors do most of the work using efficient software. These stringent performance demands tend to arise when designing for automotive systems in general, and (safety-critical) subsystems in particular. In these instances, processing must be done in real-time, with low latency and high throughput. 

However, as often highlighted in discussions about the long tail trend, these performance demands may be difficult to satisfy if architecture remains tightly coupled to particular technologies in the face of changing systemic demands. The Data-Oriented Design (DOD) approach may offer solutions to some of these problems by maximizing performance and lowering costs through optimized data layout and access patterns.

In this blog, we focus on how DOD can help address these two impediments, using the long tail concept described in the book by Richard Fabian.

What Is Data-Oriented Design (DOD)?

DOD involves designing a data domain first, then defining how objects interact within that domain, to optimize data layout and access patterns – crucial for enhancing program performance. In contrast, traditional programming paradigms such as Object-Oriented Design (OOD) are based on objects and the interactions between them without the same emphasis on optimizing data flow and access

The core principles of DOD are optimized data layout and efficient data access. Its primary goal is to organize data in a way that maximizes the performance of the hardware – in particular, the CPU and the memory subsystem – ensuring the highest possible throughput to be delivered with the lowest possible latency. DOD aligns data structures with the hardware’s optimal performance characteristics, minimizing, bottlenecks and avoiding inefficiencies that can strain the system.

A traditional Object-Oriented Design OOD design focuses on objects, their relationships, and behaviors and can lead to designs that become abstract and less efficient at handling data. This often results in data being spread throughout various memory locations, causing cache misses and inefficient utilization of the CPU’s cache structure. Data-oriented design addresses this issue by ensuring that related data are stored together thereby reducing cache misses and making better use of the CPU’s memory hierarchy.

Advantages of DOD

Here are the advantages of a data-oriented design system:

  1. Performance: Data is organized to fit optimally into cache lines, which directly impacts the efficiency of cache usage for certain computations. Additionally, well-structured access patterns ensure that data requests are handled with low latency in memory. A memory system with low latency organizes data to be accessed sequentially and contiguously, minimizing delays and maximizing throughput

  2. Scalability: DOD provides better performance as data size and algorithm complexity increase. With larger datasets, the advantages of optimized data layout and access patterns become even more significant. In a DOD system, performance scales with the growth of data, maintaining efficiency without sacrificing performance. This becomes significant in newer automotive systems, where processing large amounts of sensor data in real-time is essential.

  3. Cost Efficiency: In a typical embedded system, much of the software code focuses on data management rather than performing complex calculations. DOD optimizes the use of available hardware resources, reducing the reliance on high-end processors. This allows mid-tier processors to achieve higher performance levels that previously required more expensive, high-end processors. The resulting hardware savings offer significant cost benefits to automotive manufacturers seeking to balance performance with affordability.

Disadvantages of DOD

While there are advantages, some disadvantages need to be considered including:

  1. Learning Curve: There is a major learning effort required to make the shift from OOD to DOD. It is quite difficult for people who are used to design, such as myself. It is a real shift in paradigm to work this way.

  2. Tooling and Ecosystem: DOD has a subset of mature tooling in comparison to OOD. This often requires broader custom solutions.

  3. Hybrid Approach: In some settings, a combination of OOD and DOD might perform best and allow the best of both worlds – familiarity as well as peak performance. 

Under the Hood of a Data-Oriented Design System

There are two types of data structures for memory layout: Structure of Arrays (SoA) and Array of Structures (AoS). In an AoS layout, data is grouped by objects. Each object contains multiple fields, leading to scattered memory access when processing a specific field across all objects. This results in inefficient cache usage, as the CPU cache lines get filled with irrelevant data.

For example:

```cpp

struct SensorData {

    float position[3];

    float velocity[3];

    float distance;

};

std::vector<SensorData> sensorData;

```

In an SoA layout, data is grouped by type, enabling contiguous memory access for each field. This method significantly improves cache utilization, as the data required for computation is more likely to be loaded into the cache, reducing the number of cache misses and aligning with the long tail trend discussed in Richard Fabian’s book cited earlier.

For example:

```cpp

struct SensorData {

    std::vector<float[3]> positions;

    std::vector<float[3]> velocities;

    std::vector<float> distances;

};

SensorData sensorData;

```

By grouping data that is likely to be accessed together, DOD decreases cache misses and boosts memory access efficiency. Optimized access patterns align with the way data is processed, enabling effective prefetching and minimizing the time the CPU spends waiting for data to be loaded from memory.

For example, in computer graphics rendering applications like gaming engines, where noisy high-speed behavior with low precision is often preferred, DOD has been effectively used with massively parallel graphics processors to achieve impressive speedups. Similarly, financial systems that process large datasets in real time have also benefited from DOD, especially in large-scale deployments. These use cases indicate that DOD can be highly advantageous for systems requiring low latency and cost-effective real-time data processing.

Techniques for DOD in Software Design

Below are several techniques for applying Data-Oriented Design in software development:

  1. Identify which data elements contribute the most to cache flushes and contention. Focus on optimizing these elements before others when tuning database concurrency.

  2. Transition from Array of Structures (AoS) to Structure of Arrays (SoA) to enhance data handling efficiency, improving memory access patterns and boosting overall performance.

  3. Utilize Single Instruction Multiple Data (SIMD) instructions to enable parallel processing of data points. Use parallel processing to perform additional calculations for further computational speed and efficiency.

  4. Use tools like the Intel VTune to detect performance bottlenecks. Optimize data access patterns through tuning based on the profiling results.

Trends in DOD

Here are some current trends in Data-Oriented Design:

  1. Increased Adoption: DOD is starting to gain traction in high-performance, real-time applications due to its efficiency benefits. The enhanced performance that can be achieved is leading to widespread use in a variety of fields. 

  2. Technological Development: Responsive governance is being enabled by technologies that require massive sets of data to be handled with unprecedented efficiency. Examples include artificial intelligence (AI) and dynamic transportation security networks that enable intelligent cars.

  3. Future Outlook: As more automotive companies begin to recognize the advantages of DOD, it may soon become a standard practice in the industry in terms of their assembly lines, fitting in nicely with the long tail trend as further described by Fabian.

Conclusion

By guiding engineers to optimize code to maximize data placement and access, Data-Oriented Design (DOD) effectively addresses the twin challenges of both performance and cost in a powerful, practical way. DOD reduces reliance on expensive hardware by leveraging resource-sensitive software, leading to more efficient automotive software solutions that are more innovative, affordable, and efficient. Automotive software startups, OEMs, and Tier 1s should embrace DOD principles and tools to maximize their potential. 

Unlock the full potential of your automotive software with Data-Oriented Design. Enhance performance, reduce costs, and drive innovation in your projects. Contact us today to discover how DOD can transform your development process!

Other Articles

Key Aspects of FTTI in Automotive Safety Design

September 26, 2024

Explore the evolution of functional safety, its growing importance in industries like automotive, and the critical role of Fault Tolerant Time Interval (FTTI) in ensuring system reliability. Learn how FTTI, along with Malfunctioning Behavior Manifestation Time (MBMT) and Hazard Manifestation Time (HMT), contributes to robust safety designs, preventing hazards in safety-related systems such as ADAS and autonomous vehicles. Discover the impact of ISO 26262 standards on the development of effective fault detection and reaction mechanisms in automotive safety.

Read More »

Unleash Efficiency When Tracing Requirements

May 16, 2024

In a rapidly evolving technological landscape, the demand for systems that can not only withstand errors but also adapt to them is paramount. This article delves into the world of Fault-Tolerant (FT) systems, emphasizing their significance in maintaining the functionality and safety of critical operations across various sectors. It explores the latest advancements in FT technology, underscoring the importance of resilience and adaptability in ensuring uninterrupted service and safeguarding against potential failures.

Read More »

The Growing Need for Reliable, Adaptive, Fault-Tolerant Systems

February 29, 2024

In a rapidly evolving technological landscape, the demand for systems that can not only withstand errors but also adapt to them is paramount. This article delves into the world of Fault-Tolerant (FT) systems, emphasizing their significance in maintaining the functionality and safety of critical operations across various sectors. It explores the latest advancements in FT technology, underscoring the importance of resilience and adaptability in ensuring uninterrupted service and safeguarding against potential failures.

Read More »

Fuelling the Value of Multicast Addressing

February 6, 2024

Discover the transformative impact of Software-Defined Networking (SDN) and Multicast Addressing on automotive embedded systems. Explore how these technologies enhance communication efficiency, safety, and performance in the automotive industry, leading to cost-effective, scalable, and eco-friendly solutions. Dive into the technical advantages and practical applications for modern vehicles and infrastructure.

Read More »