Understanding Linux PAM (Pluggable Authentication Modules)

Introduction

Linux PAM (Pluggable Authentication Modules) represents a powerful framework that provides dynamic authentication support for applications and services in Linux systems. This article explores the fundamentals of PAM, its architecture, configuration syntax, and practical implementation examples.

Core Concepts

PAM operates as a middle layer between applications and authentication mechanisms. When an application needs to authenticate a user, it doesn’t need to know the details of how authentication works - it simply makes calls to the PAM framework, which handles the actual authentication process based on the system’s configuration.

Key Benefits

The modular nature of PAM provides several advantages:

  1. Applications don’t need to be modified to support new authentication mechanisms
  2. System administrators can modify authentication requirements without changing application code
  3. Multiple authentication methods can be combined flexibly
  4. Authentication policies can be standardized across the system

PAM Architecture

Service Modules

PAM divides authentication tasks into four independent management groups:

  1. account: Verifies account accessibility, expiration, and access hours
  2. auth: Handles user authentication and sets credentials
  3. password: Manages password updates and validation
  4. session: Manages tasks needed to set up and tear down user sessions

Module Types

Each management group can use various module types:

1
2
3
4
5
6
required      - Must succeed for authentication to continue
requisite - Must succeed, but fails immediately if not met
sufficient - Success is sufficient to satisfy the module requirements
optional - Module success/failure doesn't impact authentication
include - Include other configuration files
substack - Similar to include but treats the included stack as a subunit

Configuration Syntax

PAM configuration files are typically stored in /etc/pam.d/. Each service has its own configuration file. The basic syntax is:

1
type    control_flag    module_path    module_arguments

Example from /etc/pam.d/sshd:

1
2
3
4
5
6
7
8
#%PAM-1.0
auth required pam_securetty.so
auth required pam_unix.so nullok
account required pam_nologin.so
account required pam_unix.so
password required pam_unix.so nullok obscure min=4 max=8
session required pam_unix.so
session required pam_limits.so

Common PAM Modules

pam_unix.so

The standard Unix authentication module supporting standard Unix password authentication:

1
auth required pam_unix.so try_first_pass nullok

Parameters:

  • try_first_pass: Try using previously entered password
  • nullok: Allow empty passwords
  • remember=N: Remember N previous passwords

pam_ldap.so

Provides LDAP authentication support:

1
auth sufficient pam_ldap.so use_first_pass

pam_time.so

Controls access based on time:

1
account required pam_time.so

Configuration in /etc/security/time.conf:

1
2
services;ttys;users;times
login;tty*;!root;Al0800-1800

Practical Examples

Example 1: Enforcing Strong Passwords

To enforce password complexity, modify /etc/pam.d/system-auth:

1
2
password required pam_pwquality.so retry=3 minlen=12 dcredit=1 ucredit=1 ocredit=1 lcredit=1
password required pam_unix.so use_authtok sha512 shadow

This configuration:

  • Requires minimum 12 characters
  • Requires at least one digit, uppercase, special character, and lowercase
  • Allows 3 retry attempts
  • Uses SHA-512 hashing

Example 2: Limiting Login Attempts

To implement login attempt limits, add to /etc/pam.d/system-auth:

1
auth required pam_tally2.so deny=3 unlock_time=300 onerr=fail

This configuration:

  • Denies access after 3 failed attempts
  • Unlocks the account after 300 seconds
  • Fails closed (denies access) on errors

Debugging PAM

Debug Mode

Add the debug option to the module:

1
auth required pam_unix.so debug

View debug output in system logs:

1
tail -f /var/log/auth.log

Testing Configurations

Use the pamtester utility to test PAM configurations:

1
pamtester sshd username authenticate

Security Considerations

  1. Order Matters: PAM processes modules in the order listed. Ensure critical security modules are placed appropriately.

  2. Fail Secure: Use appropriate control flags to ensure authentication fails securely when modules fail.

  3. Module Parameters: Carefully consider module parameters - some can weaken security if misconfigured.

  4. File Permissions: PAM configuration files should be owned by root and not writable by others:

1
2
chmod 644 /etc/pam.d/*
chown root:root /etc/pam.d/*

Best Practices

  1. Documentation: Always document PAM changes in configuration files using comments.

  2. Backup: Create backups before modifying PAM configurations:

1
cp -r /etc/pam.d /etc/pam.d.backup
  1. Testing: Test changes in a non-production environment first.

  2. Monitoring: Regularly monitor auth logs for unusual patterns or failures.

Conclusion

Linux PAM provides a flexible and powerful framework for managing authentication in Linux systems. Understanding its architecture, configuration syntax, and proper implementation is crucial for system administrators and security professionals. Through careful configuration and following best practices, PAM can significantly enhance system security while maintaining usability.

Remember that PAM configurations can lock users out of the system if implemented incorrectly. Always maintain a root session when testing new PAM configurations, and have a backup plan for recovery if authentication breaks.

For further information, consult the official Linux-PAM documentation and system-specific guides for your distribution.

Comprehensive Guide to TEE

Introduction

Trusted Execution Environments (TEEs) are CPU-encrypted isolated private enclaves inside the memory, used for protecting data in use at the hardware level.

While the sensitive data is inside an enclave, unauthorized entities cannot remove it, modify it, or add more data to it. The contents of an enclave remain invisible and inaccessible to external parties, protected against outsider and insider threats.

As a result, a TEE ensures the following:

  • Data integrity
  • Code integrity
  • Data confidentiality

TEE terminology and operation

In a system with a TEE, we have untrusted applications running on a Rich Execution Environment (REE) and trusted applications (TAs) running on a Trusted Execution Environment (TEE).

Only trusted applications running on a TEE (Secure World) have complete access to the main processor, peripherals and memory, while hardware isolation protects these from untrusted applications running on the main operating system (Non-Secure World).

How to implement a TEE?

We could isolate applications in a “sandbox”, for example using containers. This would prevent an application from seeing and accessing data from other applications.

But what about the kernel? How to prevent a code running in kernel space from being exploited to access a certain peripheral or memory region used by a trusted application?

Software can’t protect software!

So we need support in the hardware to implement a TEE. We need a way to partition and isolate the hardware (busses, peripherals, memory regions, interrupts, etc) so that the running code does not have access to protected resources.

That’s where ARM’s TrustZone, RISC-V’s MultiZone and many other solutions come in.

Hardware support to implement a TEE:

  • ARM’s TrustZone
  • RISC-V’s MultiZone
  • AMD Platform Security Processor (PSP)
  • Intel Software Guard Extensions (SGX)
  • Apple SEP (Secure Enclave Processor)
  • Google Titan M

ARM’s TrustZone

Several commercial TEE implementations have been developed over the years:

Comprehensive Guide to HSM

Introduction

HSM, short for Hardware Security Module, is a physical device that acts as a fortress for your data, ensuring its confidentiality, integrity, and availability. HSMs are considered as standalone and tamper-resistant hardware devices specifically engineered to fortify cryptographic processes.

HSMs serve the essential functions of generating, safeguarding, and managing encryption keys, which are crucial for data encryption and decryption, and creating digital signatures and code signing certificates. HSMs can take the form of plugin cards or be integrated into various hardware components, including smart cards, appliances, and external devices.

These devices undergo rigorous testing, validation, and certification processes, adhering to strict security standards like FIPS 140-2, Common Criteria EAL 4+, or equivalent benchmarks. These benchmarks assure users that the devices offer unparalleled levels of security and shield against potential threats, ensuring the utmost confidentiality and integrity of data.

According to CerteraSSL’s HSM Comprehensive Guide, let’s first walk through the Key components of HSM, different types of HSM, features of HSM and then look into some detail of Vehicular Hardware Security Module

Key components of HSM

Key Generation and Storage

HSMs use secure random number generators (RNGs) to generate strong cryptographic keys. The generated keys are securely stored within the HSM and safeguarded against unauthorized access or extraction, ensuring keys’ integrity and randomness, which are crucial for robust cryptographic operations.

Key Management

HSMs provide a comprehensive suite of key management functionalities, which includes key generation, import, export, versioning, and facilitating key lifecycle operations like rotation and retirement.

Cryptographic Operations

HSM excels in performing various cryptographic operations. They offer encryption and decryption capabilities and play a critical role in generating and verifying digital signatures, ensuring the authenticity and integrity of electronic documents and transactions. They also support secure hashing algorithms for data integrity verification and provide secure APIs and interfaces for the seamless integration of cryptographic functionalities into applications.

Secure APIs and Interfaces

HSMs offer secure APIs and interfaces. APIs establish a secure communication channel between the application and the HSM, preventing unauthorized access and safeguarding cryptographic material from tampering.

Physical and Logical Security

HSMs incorporate robust physical and logical security measures. Physical security features include tamper-evident casings, sensors to detect physical attacks and secure key storage mechanisms. Logical security encompasses robust authentication mechanisms, access controls, and cryptographic operations executed within a trusted environment.

Compliance and Auditing

HSMs provide the necessary capabilities to comply with security regulations and industry standards. They offer auditing and logging mechanisms, enabling organizations to track and monitor cryptographic operations. Compliance features within HSMs assist in demonstrating adherence to security best practices and successfully passing regulatory audits.

Types of HSM

HSM devices come in various types, each tailored to specific use cases and industry requirements. However, there are two main types of HSMs:

General Purpose HSMs:

General purpose HSMs are versatile devices that support a wide range of encryption algorithms, such as CNG, CAPI, and more. They are designed to cater to various applications that require cryptographic services and secure key management.

General purpose HSMs are commonly used in environments where Public Key Infrastructures (PKIs) are implemented and in managing crypto-wallets and other sensitive data. These HSMs offer flexibility and compatibility with different systems and encryption standards.

Payment and Transaction HSMs:

These HSM devices are tailored to the financial industry’s requirements and are essential in maintaining compliance with Payment Card Industry Data Security Standards (PCI DSS).

Payment and transaction HSMs provide specialized cryptographic services and secure key management solutions that align with the specific needs of payment processing systems. They play a critical role in securing financial transactions and ensuring the confidentiality and integrity of payment card data.

It’s important to note that while general purpose HSMs offer broader applicability, payment and transaction HSMs focus specifically on securing payment card information and complying with industry-specific standards. The choice between these two types of HSMs depends on the specific security requirements and use cases of the organization or industry involved.

Features of HSM

HSMs offer a range of features to provide maximum security for cryptographic key management and operations. Here are some key features:

Tamper Resistance:

HSMs are built with tamper-evident casings and physical protections. They incorporate sensors and mechanisms that detect and respond to physical tampering attempts, such as opening the casing or tampering with internal components. These features ensure the integrity of the HSM and protect against unauthorized access to cryptographic keys.

Zeroization of Keys:

HSMs can securely erase cryptographic keys, ensuring their complete removal from the system and preventing unauthorized access or recovery.

Access Controls:

HSMs enforce strict access controls to prevent unauthorized users from accessing sensitive cryptographic material. Role-based access control (RBAC) mechanisms ensure that only authorized individuals or applications can interact with the HSM and perform cryptographic operations.

Robust Authentication:

HSMs employ strong authentication mechanisms to verify the identities of users or applications attempting to access the HSM. These devices mostly employ two-factor authentication, requiring a combination of something the user knows – a password and something the user has – a smart card or something the user is – biometric data to establish trust and grant access.

APIs and Interfaces:

HSMs provide secure APIs and interfaces that allow applications and systems to integrate with the HSM. Standard cryptographic protocol, are often supported, enabling seamless integration with existing systems.

Secure Design:

HSMs are designed with security in mind. They undergo rigorous testing and evaluation to meet recognized security standards, such as FIPS 140-2. The design principles encompass both physical and logical security measures to create a trusted environment for cryptographic operations.

HSM Specification

Secure Hardware Extension (SHE)

The Secure Hardware Extension (SHE) is an on-chip extension to any given microcontroller. It is intended to move the control over cryptographic keys from the software domain into the hardware domain and therefore protect those keys from software attacks. However, it is not meant to replace highly secure solutions like TPM chips or smart cards, i.e. no tamper resistance is required by the specification.

In 2019, AUTOSAR continues publish new technical requirements instead of SHE v1.1 as AUTOSAR_TR_SecureHardwareExtensions.pdf

SHE is one of the earliest examples of automotive-grade connected module hardening and quickly grew to become a standard requirement by OEMs on a global basis. It was primarily built for securing cryptographic key material against software attacks, but cannot really be used to protect communications (such as V2X). As such, it has served as a basis for later HSM standards (such as EVITA). Today, modern automotive HSMs leverage functionalities from SHE, TPM and smartcards.

SHE Background

The HIS consortium was founded in 2004 and consists of members from Audi, BMW, Daimler, Porsche, and Volkswagen to address activities and develop common standards related to automotive manufacturing. In 2006, HIS published a document describing the requirements for an HIS Security Module standard that incorporated mechanisms for error detection, authorization, and authenticity. This was further developed by ESCRYPT in partnership with OEMs Audi and BMW, and semiconductor vendors, such as Freescale (now NXP) into an open standard, publicly released in April 2009.

The resulting SHE specification outlines how a secure zone can be created within any ECU via an on-chip extension within a Microcontroller Unit (MCU), providing cryptographic services at the application layer, and isolating the storage of secret keys from the remainder of the MCU’s resources. Although the standard originated within the German automotive industry at the OEM level, it has since become an open standard accepted at the global level.

EVITA HSM

For vehicular hardware security module, we have to konw the E-safety vehicle intrusion protected applications (EVITA). EVITA is a project co-funded by the European Union within the Seventh Framework Programme for research and technological development.

The objective of EVITA is to design, verify, and prototype an architecture for automotive on-board networks where security-relevant components are protected against tampering and sensitive data are protected against compromise.

EVITA’s Background

Future automotive safety applications based on vehicle-to-vehicle and vehicle-to-infrastructure communication have been identified as a means for decreasing the number of fatal traffic accidents. Examples of such applications are local danger warnings and electronic emergency brakes. While these functionalities inspire a new era of traffic safety, new security requirements need to be considered in order to prevent attacks on these systems. Examples of such threats are forced malfunctioning of safety-critical components or the interference with the traffic flow by means of fake messages.

EVITA’s Objectives

Secure and trustworthy intra-vehicular communication is the basis for trustworthy communication among cars or between cars and the infrastructure. Therefore, the objective of the EVITA project is to design, verify, and prototype an architecture for automotive on-board networks where security-relevant components are protected against tampering and sensitive data are protected against compromise when transferred inside a vehicle.

By focusing on the protection of the intra-vehicle communication EVITA complements other e-safety related projects that focus on the protection of the vehicle-to-X communication.

Design, Implementation, and Evaluation of a Vehicular Hardware Security Module

In 2011, the 14th International Conference on Information Security and Cryptology, M. Wolf, T. Gendrullis published a paper “Design, implementation, and evaluation of a vehicular hardware security module“, introduce the vehicular HSM officially for the first time from EVITA point of view. Based on a requirements engineering approach that incorporates all security-relevant automotive use cases and all distinctive automotive needs and constraints, EVITA presented an vehicular hardware security module (HSM) that enables a holistic protection of in-vehicle ECUs and their communications.

Standardize Vehicular HSM

IT systems were never designed with security in mind. But with the increasing application of digital software and various radio interfaces to the outside world (including the Internet), modern vehicles are becoming even more vulnerable to all kinds of malicious encroachments like hackers or malware. This is especially noteworthy, since in contrast to most other IT systems, a successful malicious
encroachment on a vehicle will not only endanger critical services or business models, but can also endanger human lives. Thus strong security measures should be mandatory when developing vehicular IT systems. Today most vehicle manufacturer (hopefully) incorporate security as a design requirement.

However, realizing dependable IT security solutions in a vehicular environment considerably differs from realizing IT security for typical desktop or server environments, just porting “standard” security solutions to the, moreover, very heterogeneous IT environment usually will not work. In order
to reliably enforce the security of software security mechanisms, the application of hardware security modules (HSM) is one effective countermeasure as HSMs:

  • protect software security measures by acting as trusted security anchor,
  • securely generate, store, and process security-critical material shielded from any potentially malicious software,
  • restrict the possibilities of hardware tampering attacks by applying effective tamper-protection measures,
  • accelerate security measures by applying specialized cryptographic hardware,
  • reduce security costs on high volumes by applying highly optimized special circuitry instead of costly general purpose hardware.

Back to the paper was published, there already exist some proprietary and singlepurpose HSM realizations used, for instance, by vehicle immobilizers, digital tachographs or tolling solutions. However, these are no general-purpose, private HSMs and hence cannot be reused by other vehicular security solutions. On the other hand, general-purpose HSMs that are currently available, for instance, the IBM 4758 cryptographic co-processor, the TCG Mobile/Trusted Platform Module, or typical cryptographic smartcards are not applicable for use within an automotive security context. They, for instance, lack of cost efficiency, performance, physical robustness, or security functionality. Solely, the secure hardware extension (SHE) as proposed by the HIS consortium takes an exceptional position as it was explicitly designed for application in a automotive security context. However, the SHE module is mainly built for securing cryptographic key material against software attacks, but cannot be used, for instance, to protect V2X communications.

Security requisites (SR) and functional requisites (FR) of EVITA HSM list as follows:

  • SR.1 Autonomous, strongly isolated security processing environment
  • SR.2 Minimal immuftable trusted code to be executed prior to ECU processor
  • SR.3 Internal non-volatile memory for storing root security artifacts
  • SR.4 Non-detachable (tamper-protected) connection with ECU hardware
  • SR.5 Authentic, confidential, fresh comm. channel between HSM and ECU
  • SR.6 Autonomously controlled alert functionality (e.g., log entry, ECU halt)
  • SR.7 Only standardized, established security algorithms (e.g., NIST1 , BSI2)

  • FR.1 Physical stress resistance to endure an automotive life-cycle of ≥20 years
  • FR.2 Bandwidth and latency performance that meets at least ISO 11898 [24]
  • FR.3 Compatibility with existing ECU security modules, i.e. with HIS-SHE [21]
  • FR.4 Compatibility with existing ECU microprocessor architectures
  • FR.5 Open, patent free specifications for cost-efficient OEM-wide application

Comparison of HSMs

This can be seen in Table 5 of “Design, implementation, and evaluation of a vehicular hardware security module“(Page 16).

Trusted Platform Module

Due to the massive use of x86 chips in vehicle, Trusted Platform Module (TPM) was is and will be used in E/E architecure to protect user data.

Back up to the days that hackers can quickly weaponize the valuable data in laptops and computers, Trusted Computing Group (TCG) develops and prompts specification and standards of TPM to against these threat. TPM is a secure crypto-processor which is attached to a device to establish secure operations. Using a TPM helps to protect a user’s identity and sensitive data by storing the relevant keys vital for encryption, decryption and authentication. Doing so provides a first line of defense against potentially critical malware and firmware attacks, as it ensures all data remains encrypted even if an attack takes place.

Solutions for virtual environments of TPM

vTPM

A vTPM is a software-based representation of a traditional TPM 2.0 chip. It carries out the same hardware-based security functions a TPM, for example, attestation, key and random number generation – without the physical chip being required.

AutoSar IDS Architecture

Here,

  • Doc Name refer to docs that AutoSar published.
  • Section as item to its parent.
  • Definition The definition to important concept.

AutoSar IDS Reference Documents

Domain Documents Description
IdsM AUTOSAR_RS_IntrusionDetectionSystem.pdf This document specifies requirements for the AUTOSAR Intrusion Detection System (IDS) .
IDS Protocol Layer AUTOSAR_PRS_IntrusionDetectionSystem.pdf This Protocol Requirements Specification defines the format, message sequences and semantics of the AUTOSAR Protocol IDS
IdsM specification AUTOSAR_SWS_IntrusionDetectionSystemManager.pdf This specification describes the functionality, API, and the configuration for the AUTOSAR Basic Software module Intrusion Detection System Manager ( IdsM).

General Architecture of a distributed onboard IDS

An onboard IDS according to the AUTOSAR standard consists of the following elements

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
graph BT;
subgraph In-Vehicle
subgraph ECUA
A(Host Sensor)-->C(IdsM)
B(Ethernet Sensor)-->C(IdsM)
I(Sem)---C
end
subgraph ECUB
E(Host Sensor)-->G(IdsM)
F(Ethernet Sensor)-->G(IdsM)
J(Sem)---G
end

C-->D(IdsR)
G-->D
end

subgraph TSP
H
end

D-->H(SOC)

style D stroke:#333,stroke-width:2px,stroke-dasharray: 5 5;

As listed, we got the following elements that constitute AutoSar IDS standard:

  • Secure Sensor
  • Intrusion Detection System Manager(IDSM)
  • Security Event Memory(Sem)
  • Intrusion Detection System Repoter(IDSR)

The Intrusion Detection System Manager (IdsM) is a Basic Software module (for the AUTOSAR Classic Platform) or a Platform Service (for the AUTOSAR Adaptive Platform) that collects and centrally aggregates security incidents that possibly result from malicious attacks on the vehicle’s software, communications or electronics system. In each of the security relevant ECUs or machines within the vehicle, an instance of the IdSM module or service collects and filters security events (optionally including additional data) in order to store them in a local Security Event Memory (Sem) and/or to forward them over the vehicle network to a central Intrusion Detection System Reporter (IdsR). This IdsR might be, for example, located within a telematics unit enabling it to send security reports and associated data via a cellular network to an OEM’s Security Operations Center (SOC). This information is then analyzed by the Security Incident and Event Management (SIEM) and, if necessary, used to develop and decide on appropriate defense or mitigation actions to counter the attack.

Security Sensors and Security Events

AUTOSAR BSW modules, CDD and SWC can act as Security Sensors. Security sensors report Security Events (SEv) to the IdsM, and the module IdsM itself can also be used as a Security Event sensor.

IDSR

IdsR is the bridge between Car and Backend, which is NOT SPECIFIED by AutoSar actually. The IdsR should typically further enrich the received data e.g. geo-position based on OEM requirement for the needs of SIEM.

IDS protocol

This Protocol Requirements Specification AUTOSAR_PRS_IntrusionDetectionSystem defines the format, message sequences and semantics of the AUTOSAR Protocol IDS.

The main use case for the IDS protocol is the propagation of Qualified Security Events QSEv to the IdsR in a way that is independent from the kind of ECU or the used communication mechanism.

Protocol specification

Overview

Event Frame Timestamp(optional) Context Data(optional) Signature(optional)

Protocol Detail

FieldName Length Purpose Description
Protocol Version 4 Bit The version of the IdsM protocol
Protocol Header 4 Bit IdsM protocol header information: Bit 0: 0 - No Context Data included, 1 - Context Data included
Bit 1: 0 - No Timestamp included, 1 - Timestamp included
Bit 2: 0 - No Signature included, 1 - Signature included
Bit 3: reserved
IdsM Instance ID 10 Bit Unique identifier of the sending, IdsM - Instance 0 - 1023
Sensor Instance ID 6 Bit Identifier to differ between multiple instances of sensors
Event Definition ID 16 Bit Unique identifier of a Security Event Range of AUTOSAR internal IDs: 0…0x7FFF
Range of Customer specific IDs: 0x8000…0xFFFE
Count 16 Bit Number of IdsM calls which result in the current event after processing the configured filter, e.g. EventAggregation 1.. 65535 .
Reserved 8 Bit Reserved for future use
Timestamp 64 Bit Timestamp/Timestamp when event was detected: (optional) Byte0 - Bit7: 0 - AUTOSAR Standard, 1 - Auxiliary/ OEM specific
Byte0 - Bit 6: reserved
Resolution in ms. Maybe not necessary for every event type (optional).
Context Data 2..(2^31 + len) Bytes Binary blob attached by the sensor. (optional) Most Significant Bit of first byte signals if ContextData Length is encoded in
7 Bit (1 Byte) or 63 Bit (4 Bytes):
Context Data [0]: MSB = 0 - 7 Bit Length Information encoded in Context Data [0]: 1..127
Context Data [0]: MSB = 1 - 63 Bit Length Information encoded in Context Data [0..3]: 1..2^31
len = 1 or 4
Signature 3…65537 Bytes Signature for encryption of security event. (optional) Signature calculated with Event Frame + optional Timestamp + optional Context Data
Signature[0..1]: Signature Length 1..65535

Byte Order The IDS protocol uses big endianess as byte order also known as Motorola format

Event Frame

Event Frame consists of Protocol Version, Protocol Header, IdsM Instance ID, Sensor Instance ID, Event Definition ID, Count and Reserved part.

Timestamp
Source Reserved Nanoseconds Seconds

Bit 7: Souerce

  • 0: AUTOSAR Standard CP: StbM - AP: ara::tsync
  • 1: Auxiliary/ OEM Specific timestamp

If Bit 7 = 0, then:

  • Reserve:Bit 6
  • Nanoseconds:Byte 0(0::5)::3
  • Seconds: Byte 4::7
Context Data

The IDS protocol provides an optional feature to enrich the standard security event transfered in the Event Frame with more detailed information. Therefore context data can be added. It is a binary blob attached by the sensor. These data includes specific detailed information about the security event which can be used by the SOC for improved analysis of the security incident, e.g. a malformed message detected by a communication sensor.

IdsM has no knowledge of the content or structure of these data. Only the issuing sensor and the Backend or SOC knows it

Signature

The IDS protocol provides an optional feature to make the transmission of QSEv more secure. A digital signature can be added to the IDS message. It can be used to ensure authenticity as well as to prove integrity of signed messages from the IdsM via all communication systems until reaching the Backend or SOC (End2End-Security).

IDSM

The Intrusion Detection System Manager handles security events reported by security sensors.

  • IDSM privides a standardized interface for receiving SEvs. SEv can be reported with optional context data.
  • IDSM has the capability of qualifying SEvs, filters and transforms reported SEVs into QSEvs
  • QSEvs can be stroed locally on Security Event Memory(Sem) or forwarded towards configured sinks, or both.
  • Sinks could be Diagnostic Event Manager(Dem) and IDSR. Both may pass QSEv data to SOC.

The IdsM shall provide interfaces for reporting SEv. The interfaces shall allow to report the following SEv properties:

  • Security event type: Uniquely identifies the security event type
  • Context data: Optional data which can be used to analyze the security event in the SOC.
  • Timestamp: Optional timestamp provided by the sensor.
  • Count: Optional count provided by the sensor.

IdsM shall provide mechanisms to add timestamps to SEvs.

Timestamp Sources: IdsM shall provide mechanisms to let application or sensor software provide timestamps.

IdsM shall allow to transmit the QSEv and context data by using an IDS protocol, which is independent from the underlying bus technology

The IdsM shall be able to locally persist QSEvs via a user defined diagnostic memory. The user defined diagnostic memory shall be separate from the main diagnostic memory to allow separate access control and protection of the NVM block which is used to store the QSEv records. Persisted QSEvs can be accessed at a later point in time for analysis without relying on, e.g., network connectivity.

IdsM shall support limiting the rate of QSEvs transmitted to the IdsR and the bandwidth consumed by these transmissions.

AUTOSAR_TPS_SecurityExtractTemplate

The Security Extract Template (SECXT) is part of the Intrusion Detection System (IDS). In the context of ECU development projects, the SECXT serves multiple use cases. The SECXT specifies the security events and their properties for a vehicle on system level. The Security Extract as a specific, “standalone” file for security event definitions is in particular useful in view of the reasonable expectation that new approaches or kinds of attacks are identified after SOP of a vehicle. The resulting new or changed security events lead to an updated SECXT file that can subsequently be deployed onto the affected ECUs or machines of a vehicle together with a software update. Additionally, the SECXT file can potentially be used by the SIEM and SOC to interpret incoming reports of the IdsR instances of the vehicles in field.

To summarize, the Security Extract Template defines a standardized AUTOSAR exchange format for defining security events and their properties. The Security Extract (SECXT) is formalized as an ARXML file and applicable for both the AUTOSAR Adaptive and AUTOSAR Classic Platforms in a way similar to a Diagnostic Extract file.

Conceptual Background

Main Development Phases for an IDS

Typically, an Intrusion Detection System (IDS) is based on the system parts IdsM, IdsR and the Security Operation Center (SOC) as exemplarily depicted in General Architecture of a distributed onboard IDS

The development of such an IDS can be divided into the following main phases:

  1. Security Analysis phase
  2. IDS Design phase
  3. IDS Deployment phase
  4. IDS Operational phase

The Security Extract Template supports all these four phases and can both be used for specification and exchange of IDS related definitions by and between OEMs and their suppliers. Therefore, a Security Extract file has potentially a high number of release cycles starting with security analysis and ending with “end of support” for a specific vehicle.

Security Analysis Phase

In the Security Analysis phase, the vehicle’s electronics and software system is examined and analyzed by security experts to identify and evaluate potential approaches of attacks on the components of the system that could lead to a security breach. In a second step, based on these potential attack approaches, detectable events that deviate from the normal behavior of the system are identified and defined as Security Events.

IDS Design Phase

The IDS Design phase distributes, customizes and adapts the generic IDS components towards a concrete vehicle electronics and software system taking into consideration the security events identified in the previous phase

In this phase, the Security Extract Template is enriched with the design decisions such as definition of IdsM instances, the mapping of security events onto them and the configuration of filters

IDS Deployment Phase

The IDS Deployment phase comprises the realization of the IDS Design from the previous step towards the real system in hardware and software.

This phase is supported by the Security Extract Template through definition of IdsM instance deployment onto specific ECU-HW and the possibility to derive ECU configuration parameters for the IdsM modules on the Classic Platform

IDS Operational Phase

The IDS Operational phase refers to the running IDS in the field when the vehicle is used by the end customer.

This phase is still regarded as part of the development process because it typically involves an IDS update process to keep the IDS up to date with new versions of application and platform software as well as with newly identified attack approaches and thus new security events.

During the IDS update process, Security Extract files can be used to reconfigure the IdsM instances of the IDS and also to make these reconfigurations known to the IdsR.

This is a notable difference to other AUTOSAR (M2 level) exchange files (e.g. System Description) which usually do not evolve further after the final configuration of the ECU-HW devices of the vehicle has been specified for SOP. On the other hand, the Security Extract file is expected to be maintained and further extended even after SOP of the vehicle it relates to due to its involvement in the IDS update process.

Abbreviations

Abbr Description
API Application Programming Interface
BSW Basic Software
BswM Basic Software Mode Manager
CDD Complex Device Driver
Csm Crypto Service Manager
Dcm Diagnostic Communication Manager
Dem Diagnostic Event Manager module
DET Default Error Tracer
ECU Electronic Control Unit
ECUC ECU configuration
ID Identifier
IDS Intrusion Detection System
IdsM Intrusion Detection System Manager
IdsR Intrusion Detection System Reporter
IF Interface
MCU Microcontroller Unit
NvM Non-volatile memory
NVRAM Non-volatile random access memory
OEM Original Equipment Manufacturer
PDU Protocol Data Unit
PDU ID PDU Identifier
PduR PDU Router
QSEv Qualified Security Event
RTE Runtime Environment
SecXT Security Extract
Sem Security Event Memory
SEv On-board Security Event
SIEM Security Incident and Event Management
StbM Synchronized Time-Base Manager
SW-C Software Component
SOC Security Operation Center
TP Transport Protocol
BSW Basic Software
CDD Complex Device Driver
SWC Software Component
SWS Software Specification
SOC Security Operation Center

Comprehensive Guide to Diagnostic

Introduction

Diagnostic related history and concept, from OBD, UDS to DoCAN, DoIP etc.

OBD

OBD stands for On-Board Diagnostics and is a computer system inside of a vehicle that tracks and regulates a car’s performance. This on-board computer system collects information from the network of sensors inside the vehicle, which the system can then use to regulate car systems or alert the user to problems. A technician can then simply plug into the OBD system to collect vehicle data and diagnose the problem. OBD systems have been a great help in helping users better understand vehicle diagnostics.

Based on ISO15765-1:2011(DoCAN) and ISO 13400-1:2011(DoIP), enhanced and legislated WWH-OBD diagnostic specifications applicable to the OSI layers are divided into:

ISO 7498-1/ISO 10731
OSI 7 layers
Vehicle manufacturer
enhanced diagnostics
Legislated OBD Legislated WWH-OBD
Application Layer ISO 14229-1,
ISO 14229-3
ISO 15031-5 ISO 27145-3,
ISO 14229-1
Presentation (layer 6) Vehicle manufacturer specific ISO 15031-2,
ISO 15031-5,
ISO 15031-6,
SAE J1930-DA,
SAE J1979-DA,
SAE J2012-DA
ISO 27145-2,
SAE 1930-DA,
SAE J1979-DA,
SAE J2012-DA,
SAE J1939:2011, Appendix C (SPN),
SAE J1939-73:2010, Appendix A (FMI)
Session (layer 5) ISO 14229-2 ISO 14229-2 ISO 14229-2
Transport protocol (layer 4) ISO 15765-2,
ISO 13400-2
ISO 15765-2,
ISO 15765-4
ISO 15765-4,
ISO 15765-2, 
ISO 27145-4,
ISO 13400-2
Network (layer 3) ISO 15765-2,
ISO 13400-2
ISO 15765-2,
ISO 15765-4
ISO 15765-4,
ISO 15765-2,
ISO 27145-4,
ISO 13400-2
Data link (layer 2)
Physical (layer 1)
ISO 11898-1,
ISO 11898-2,
ISO 11898-3,
ISO 11898-5,
ISO 13400-3,
or user defined
ISO 11898-1,
ISO 11898-2,
ISO 15765-4
ISO 15765-4,
ISO 11898-1,
ISO 11898-2,
ISO 27145-4,
ISO 13400-3

Components

A basic OBD system consists of a central system, a network of sensors, a connection point and indicators, creating a complete monitoring system with standardized access and readability. The OBD system consists of the following components:

  • ECU: The central part of the OBD system is the Electronic Control Unit, or ECU. The ECU collects input from various sensors throughout the vehicle. The ECU then uses this data to either control parts of the vehicle, like fuel injectors, or monitor for issues.
  • Sensors: There are sensors throughout vehicles covering every area from the engine and chassis to the electronic system itself. Each one of these systems sends codes to the ECU, specifying the source and the parameters of the signal. The ECU then “reads” and interprets this signal.
  • DTC: If a sensor sends information to the ECU that falls outside of the normal range, the ECU saves the information as a code called a Diagnostic Trouble Code, or DTC. The DTC code essentially is a list of letters and numbers, which indicate the source and nature of the problem. DTC codes are usually standardized but may be manufacturer-specific. When a DTC is saved, the ECU sends a signal to your indicator light to state that a problem has been found. The DTC can also be pulled by linking a sensor to the connector for the OBD system.
  • MIL: When the ECU collects a DTC code, it sends a signal to the vehicle dashboard to turn on the appropriate indicator lights. These lights, known formally as Malfunction Indicator Lights or MILs, provide an early warning system for vehicle malfunctions. Generally speaking, if the light turns on and stays on, the problem is minor. If the light flashes, the problem is urgent.
  • DLC: All of the data and DTC codes collected by the ECU can be accessed via the Diagnostic Link Connector or DLC. The DLC port is the point of access for vehicles with OBD systems and is often found beneath the dashboard on the driver’s side of the vehicle, though it may be located elsewhere in commercial vehicles. Current vehicles are made with a standard OBDII system so that any scan tool with a type 2 cable can connect to the type 2 connector.

History

The history of on-board diagnostics goes back to the 1960s. Several organizations set the groundwork for the standard, including the California Air Resources Board (CARB), the Society of Automotive Engineers (SAE), the International Organization for Standardization (ISO) and the Environmental Protection Agency (EPA).

It’s important to note that before standardization, manufacturers were creating their own systems. The tools from each manufacturer (and sometimes models from the same manufacturer) had their own connector type, electronic interface requirements. They also used their own custom codes for reporting problems.

1968 — The first OBD computer system with scanning capability was introduced by Volkswagen.

1975 — Datsun introduced a simple OBD system with limited non-standardized capabilities.

1979 — The Society of Automotive Engineers (SAE) recommends a standardized diagnostic connector and set of diagnostic test signals.

1980 — GM introduced a proprietary interface and protocol capable of providing engine diagnostics through an RS-232 interface or more simply, by flashing the Check Engine Light.

1988 — Standardization of on-board diagnostics came in the late 1980s after the 1988 SAE recommendation that called for a standard connector and set of diagnostics.

1991 — The state of California required all vehicles to have some form of basic on-board diagnostics. This is referred to as OBD I.

1994 — The state of California mandated that all vehicles sold in the state starting in 1996 must have OBD as recommended by SAE — now referred to as OBDII. This stems from the desire to perform across the board emissions testing. OBDII included a series of standardized diagnostic trouble codes (DTCs).

1996 — OBD-II becomes mandatory for all cars manufactured in the United States.

2001 — EOBD (European version of OBD) becomes mandatory for all gasoline vehicles in the European Union (EU).

2003 — EOBD becomes mandatory for all diesel vehicles in the EU.

2006 — All vehicles manufactured in Australia and New Zealand were required to be OBD2 compatible.

2008 — Starting in 2008, all vehicles in the US are required to implement OBDII through a Controller Area Network as specified by ISO 15765-4.

2012 — ISO 27145 consists of the following parts, under the general title Road vehicles Implementation of World-Wide Harmonized On-Board Diagnostics (WWH-OBD) communication requirements

2013 — The ISO 13400 series has been established in order to define common requirements for vehicle diagnostic systems implemented on an IP communication link

2021 — SAE J1979-2 (“OBDonUDS”) , describes the communication between the vehicle’s OBD systems and test equipment required by OBD regulations. The new standard will be introduced in the USA from 2023 and will be mandatory from 2027

UDS (ISO 14229)

The Automotive UDS Protocol, also known as Unified Diagnostic Services, is a communication protocol used in the automotive industry for diagnostics, debugging, and vehicle communication. It provides a standardized way for electronic control units (ECUs) in vehicles to communicate with diagnostic tools and software applications. The protocol is based on the ISO 14229 standard and is widely used by automotive manufacturers and service technicians.

ISO 14229 has been established in order to define common requirements for diagnostic systems, whatever the serial data link is.

To achieve this, ISO 14229 is based on the Open Systems Interconnection (OSI) Basic Reference Model in accordance with ISO/IEC 7498-1 and ISO/IEC 10731, which structures communication systems into seven layers. When mapped on this model, the services used by a diagnostic tester (client) and an Electronic Control Unit (ECU, server) are broken into the following layers in accordance with below table:

  • Application layer (layer 7), unified diagnostic services specified in this document, ISO 14229-3 UDSonCAN, ISO 14229-4 UDSonFR, ISO 14229-5 UDSonIP, ISO 14229-6 UDSonK-Line, ISO 14229-7 UDSonLIN, ISO 14229-8 UDSonCXPI, further standards and ISO 27145-3 VOBD.
  • Presentation layer (layer 6), vehicle manufacturer specific, ISO 27145-2 VOBD.
  • Session layer services (layer 5) specified in ISO 14229-2.
  • Transport layer services (layer 4), specified in ISO 15765-2 DoCAN, ISO 10681-2 Communication on FlexRay, ISO 13400-2 DoIP, ISO 17987-2 LIN, ISO 20794-3 CXPI, ISO 27145-4 VOBD.
  • Network layer services (layer 3), specified in ISO 15765-2 DoCAN, ISO 10681-2 Communication on FlexRay, ISO 13400-2 DoIP, ISO 17987-2 LIN, ISO 20794-3 CXPI, ISO 27145-4 VOBD.
  • Data link layer (layer 2), specified in ISO 11898-1, ISO 11898-2, ISO 17458-2, ISO 13400-3, IEEE 802.3, ISO 14230-2, ISO 17987-3 LIN, ISO 20794-4 CXPI, and further standards, ISO 27145-4 VOBD.
  • Physical layer (layer 1), specified in ISO 11898-1, ISO 11898-2, ISO 17458-4, ISO 13400-3, IEEE 802.3, ISO 14230-1, ISO 17987-4 LIN, ISO 20794-4 CXPI, and further standards, ISO 27145-4 VOBD.
ISO 7498-1/ISO 10731
OSI 7 layers
Vehicle manufacturer
enhanced diagnostics
VOBD
Application Layer ISO 14229-1,
ISO 14229-3 UDSonCAN,
ISO 14229-4 UDSonFR,
ISO 14229-5 UDSonIP,
ISO 14229-6 UDSonK-Line,
ISO 14229-7 UDSonLIN,
ISO 14229-8 UDSonCXPI,
further standards
ISO 27145-3
Presentation (layer 6) Vehicle manufacturer specific ISO 27145-2
Session (layer 5) ISO 14229-2 ISO 14229-2
Transport protocol (layer 4) ISO 15765-2,
ISO 13400-2
ISO 27145-4
Network (layer 3) ISO 15765-2,
ISO 13400-2
ISO 27145-4
Data link (layer 2)
Physical (layer 1)
ISO 11898-1,
ISO 11898-2,
ISO 11898-3,
ISO 11898-5,
ISO 13400-3,
or user defined
ISO 27145-4

UDS Protocol

The UDS protocol is a diagnostic protocol used in Automotive Vehicles to find the cause of a problem for the health check. Basically, it is used in the automotive field for vehicle diagnostic, ECU new software flashing, etc. Nowadays the use of the protocol is increasing due to its flexibility. This protocol is defined in ISO-14229-1 standard and it is derived from the ISO 14230-3 (KWP-2000) and ISO 15765-3 (Diagnostic Communication over the CAN (DoCAN).

The UDS (Unified Diagnostic Services) protocol operates at the 5th (Session layer) and 7th (Application layer) of the OSI model.

UDS Protocol Architecture

Those services allow a tester (client) to control diagnostic functions in an on-vehicle Electronic Control Unit (server) applied for example on the electronic fuel injection, automatic gearbox, anti-lock braking system, etc., connected on a serial data link embedded in a road vehicle.

The server, usually a function that is part of the ECU, uses the application layer services to send response data, provided by the requested diagnostic service back to the client. The client is usually referred to as an External Test Equipment when it is off-board but can in some systems, also be an on-board tester. The usage of the application layer services is independent of the client being an off-board or on-board tester. It is a possibility to have more than one client on the same vehicle system.

The most typical network configuration of the client-server communication for the vehicle diagnostics: the client as an Off-board tester. Communication is based on a request-response model. In the context of diagnostics, the following concepts are useful for a better understanding of the semantics handled on the UDS standard environment:

  1. Diagnostic Trouble Codes (DTC ): The numerical common identifier fault condition identified by the on-board diagnostic system.
  2. Diagnostic Data: Data that is located in the memory of an electronic control unit that may be inspected and/or possibly modified by the tester (diagnostic data includes analogue inputs and outputs, digital inputs and outputs, intermediate values and various status information). EXAMPLES: vehicle speed, throttle angle, mirror position, system status, etc.
  3. Diagnostic Session: The current model of the server, which affects the level of diagnostic functionality.
  4. Diagnostic Routine: The routine that is embedded in an electronic control unit and that may be started by a server upon a request from the client. NOTE: It could either run instead of the normal operating program or run concurrently to the normal operating program. In the first case, the normal operation of the ECU is not possible. In the second case, multiple diagnostic routines may be enabled that run while all other parts of the electronic control unit are functioning normally.
  5. Tester: The system that controls functions such as test, inspection, monitoring or diagnosis of an in-vehicle electronic control unit and which may be dedicated to a specific type of operator (e.g. a scan tool dedicated to garage mechanics or a test tool dedicated to the assembly plant agents).

UDS Protocol Frame Format

UDS is a Request and Response-based protocol based on client-server architecture, and it has having unique service ID(SID). SID is the size of one byte, and it ranges from 0x00 to 0x3E .

Basically, there are 4 types of frame formats:

  1. Request frame with sub-function ID
  2. Request frame without sub-function ID
  3. Positive Response Frame
  4. Negative Response Frame

Use 0x11(ECU Reset) service as an example:

SID Sub-function ID Description
0x11 0x01 Hard Reset
0x11 0x02 Key-Off On Reset
0x11 0x03 Soft Reset

Request Frame Format

Service ID (SID) Sub-Function ID (optional) Data Parameters

Response Frame Format

Positive Response

In UDS diagnostics, the tester acts as a client, and ECUs act as a server. When the server (ECU)receives the service request from the tester, the server checks the message. If everything is fine, then it executes the requested service and responds to the client with a positive response. If the response is positive, then the 6th bit of the SID should be 1 .

For example,

Service ID – 0x31 => 0 0 1 1 0 0 0 1

For a positive response, 0 1 1 1 0 0 0 1 is equal to 0x71 (0x31 + 0x40).

In another way, we can say the positive response means SID+ 0x40 , There is no logical reason for this. Simply, it is defined in International standard IS0-14229-1:

Service ID (SID) + 0x40 Sub-Function ID (optional) Data Response Code

Negative Response

In UDS diagnostics, the tester acts as a client, and ECUs act as a server. When the server (ECU) receives the service request from the tester, ECU checks the message. If the server finds something wrong, then it executes the negative response and sends the Negative response code(NRC). There are some Negative Response Codes given below.

  1. General Rejection – 0x10
  2. Sub-Function Not Supported – 0x12
  3. Incorrect Message Length(IML) – 0x13
  4. Busy Repeat Request – 0x21
  5. Condition not correct – 0x22
  6. Request sequence Error – 0x24
  7. Request Out Of Range(ROOR) – 0x31
  8. Security Access Denied – 0x33
  9. Invalid Key – 0x35
0x7F Service ID Negative Response Code

Functions of Diagnostic Services In UDS Protocol

Besides specifying services’ primitives and protocols that describe the client-server interaction, UDS also defines within its framework several functional units that comprise several services each, identified with a hexadecimal code. These units are intended for the different individual purposes that support the overall diagnostic function/task. The UDS protocol having different services for the different types of work tasks to do on the server. These are having 6- types as:

  1. Diagnostic and communication management.
  2. Data Transmission.
  3. Stored Data Transmission.
  4. Input/Output Control.
  5. Remote activation of routine.
  6. Upload/Download.

Each functional group has more than one service ID for different-2 tasks so to get the detail of the above functional group and related services.

Diagnostic Session Control

The diagnostic session’s service ID is **0x10 **and the Response SID is 0x50 . Diagnostic Session control is used for controlling the diagnostic session of the ECU. This service is used to change the current diagnostic session to a different session. These sessions are used to enable or disable a particular set of diagnostic functions and features in an ECU.

The main purpose of this diagnostic session is to give security to the ECU. It will prevent the ECU from unwanted access. And only an authorized person can access it. If every diagnostics service can access the ECU, it may get damaged by the wrong flashing of software.

So before making any diagnostic request, the Client must make sure that this service is accessible or not in the ECU current session. if not, then first send a request for session change after that desirable request shall be performed.

If the Server(ECU) is in a non-default session or if there is diagnostic inactivity for 5 seconds, the ECU goes back to the default session.

Sub-function Description
Default Session(0x01) * This session is in an idle state. Whenever the ECU is powered ON, it will be in this default session only. ECU remains in default session until another diagnostic session is requested from the client.
* Services like Write data byte identifier (0x2E), Read data byte identifier (0x22), ECU Reset (0x11), Tester Present (0x3E), and Reading DTC (0x19) are available in this session.
* Security access (0x27) is not available. This service has low security compared to all other diagnostics session control sub-functions.
Example : In a garage, a person is trying to read the Diagnostic Trouble Code(DTC).
Programming Session(0x02) * This Session is used to program the ECU or transfer data from client to server.
* All the services that are allowed in the Default session are allowed in this session.
* ECUs are entered into default session if this session is ended or expired.
Example : Programming session is End line engineer flashing calibration software.
Extended Session(0x03) * All services allowed in the Default session are allowed in this session
* This session is used to unlock the additional diagnostic functions also
* Security access(0x27) is allowed in this session, meaning security levels are unlocked in this session
* ECUs entered into default session if this session is ended or expired
Example : This session is End of line engineers doing a dynamic vehicle test to check the security level
Safety system Diagnostic system(0x04) * Used to test all safety-critical diagnostic functions.
* This diagnostics session control sub-function has a high level of security.
Example : This session is checking the safety of Airbags, and tire pressure monitors.
0x05 to 0x3F Reserved
Vehicle Manufacture Specific (0x40 to 0x5F) This session depends on Each OEM, If they want to implement any session based on their requirement they can use it.
System supplier Specific (0x60 to 0X7E) This session is also like Vehicle Manufacture Specific only but instead of the vehicle manufacturer, if the system suppliers want to implement any session based on their specific requirement they can use it.
0x7F Reserved

ECU Reset

The ECU reset’s service ID is 0x11 and the Response SID is 0x51 .

Sub-function Description
Hard Reset (0x01) * Hard reset means removing the battery (power supply) from the ECU and connecting the ECU again with the battery.* In this type of reset, ECU re-initializes the core hardware components of the system and also It will re-initializes the Non-volatile and volatile memory.* This reset may lose some data because the battery is removed suddenly during the running time of the ECU.
Key off On Reset (0x02)) * The Key Off-On Reset is simply the ignition Off-On process of a vehicle. It is the normal sleep-wake-up mode of a Microcontroller.* When we are doing Key Off On reset ECUs will not get the power down immediately. It will go to the boot mode then it will store all the data in Non-Volatile memory and de-initialize the hardware variables without losing any data.* In this method of resetting, there is no chance of losing data. This is the proper way to reset, and most of the OEMs are using this type of ECU Reset.
Soft Reset (0x03) * A soft reset is nothing but restarting the application’s main software. When we do this type of reset, the stack pointer of the microcontroller points to the**main()** function’s address. Then it will start to execute from first.For Example, will take a watchdog reset. Whenever hanging or any malfunctions are happening, this watchdog timer will reset the microcontroller and it will start from the main() function.
Enable Rapid Power shutdown (0x04) * In this type of reset, ignition off will not occur. ECU will go to sleep mode and ECU is ready to wake up at any time.
Disable Rapid Power shutdown (0x05) * This service is used to disable the previously enabled rapid power shutdown.

Security Access

The Security Access’s service ID is **0x27 **and the Response SID is 0x67. This service will grant Read, and Write access to the particular service. Before processing any of the Service IDs(SID) operation, it’s mandatory to check the security access, to know whether, for this SID, the client has access to read or write.

Security access is granted based on these two:

  1. Seed (0x01)
  2. Key (0x02)
  • The tester sends the request to unlock the ECU using SID 0x27 and sub-function ID 0x01 . 0x01 means requesting for seed.
  • The UDS server receives the request assumes conditions are correct and generates the random seed and key based on a cryptographic algorithm and the Server sends the Seed to the client with a positive response.
  • With the received seed, the tester tool generates the key and sends this key to the server to unlock the ECU using SID 0x27 and sub-function ID 0x02 . 0x02 means key.
  • If the unlock key sent by the tester tool(client) matches with the server expecting key it will send the positive response and Unlock the ECU otherwise it will send a negative response with the specific negative response code.

Authentication Service

This Authentication service was added after the year 2020, and this is used to provide a standardized approach to more modern methods of authentication than are permitted by the Security Access ( 0x27 ) service, including bidirectional authentication with PKI-based Certificate Exchange.

The Authentication’s service ID is 0x29 and the Response SID is 0x69 .

ReadMemoryByAddress (0x23): UDS Protocol

The ReadMemoryByAddress service enables the client to request memory data from the server. It involves specifying a starting address and the size of memory to be read.

The ReadMemoryByAddress request message is used to request memory data from the server based on the specified memory address and size. The byte count for the memory address and size is determined by the addressAndLengthFormatIdentifier (low and high nibble). Using a fixed addressAndLengthFormatIdentifier is also an option. Any unused bytes in the memory address or size parameter are padded with the value 00 hex in the higher range address locations.

WriteMemoryByAddress (0x3D): UDS Protocol

This service allows the client/Tester to write information into the server/ECU at one or more contiguous memory locations. The tester sends a memory address, and the number of bytes, and a data string (according to the number of bytes ). The ECU writes the data string into its memory. The addressAndLengthFormatIdentifier parameter in the request specifies The number of bytes used for the memory address and memory size parameter.

Routine Control (31 hex): UDS Protocol

The Vehicle Diagnostics may require testing the faulty component in a given range of parameters. During vehicle testing, certain system tests may need an extended runtime. To initiate a test, the client triggers a routine in the server’s memory. Two methods exist in this UDS Protocol remote request service: one where the client interrupts the routine to halt it, and the other where the server/ECU completes the routine after a specified time frame. This service allows the client to initiate, interrupt, and check the result of a routine after successful execution.

Requests: RoutineControlType:

  1. startRoutine (01 hex).
  2. stopRoutine (02 hex).
  3. reqestRoutineControl (03 hex).

Other OBD and UDS Ddiagnostic Services

One of the most important concepts in UDS is the diagnostic service identifier (SID), which is a one-byte code that specifies what kind of service the tester is requesting from the ECU.

SID Description
01-0Ahex OBD services
10hex DiagnosticSessionControl
11hex ECUReset
14hex ClearDiagnosticInformation
19hex ReadDTCInformation
22hex ReadDataByIdentifier
23hex ReadMemoryByAddress
27hex SecurtityAccess
29hex Secure Diagnostics
2Ehex WriteDataByIdentifier
2Fhex InputOutputControlByIdentifier
31hex RoutineControl
34hex RequestDownload
35hex RequestUpload
36hex TransferData
37hex RequestTransferExit
38hex RequestFileTransfer
3Dhex WriteMemoryByAddress
3Ehex TesterPresent
85hex ControlDTCSetting
86hex ResponseOnEvent
87hex LinkControl

ODX (Open Diagnostic data eXchange)

The standard ODX ISO 22901-1 (O pen D iagnostic data e X change) describes an XML format for exchanging diagnostic data. It contains both diagnostic specifics and flash data for individual ECUs, as well as Information on accessing the entire vehicle network. The data is usually processed on a D-Server in accordance with ISO 22900-3.

PDX (Packaged ODX), a collection of CDX

Timing Performance

Timing parameters play a crucial role in Unified Diagnostic Services (UDS) for managing the timing aspects of diagnostic communication between electronic control units (ECUs) in vehicles and diagnostic tools. These parameters help regulate the timing of various diagnostic services and functions within the UDS protocol to ensure efficient and reliable communication. Some common timing parameters in UDS include:

  1. P2 Timeout : This parameter defines the time interval between consecutive diagnostic requests from the diagnostic tool to the ECU. It ensures that the diagnostic tool does not overwhelm the ECU with requests.
  2. P2 Timeout *: This parameter is similar to P2 Timeout but is used for specific services that require a different timing interval.
  3. P3 Timeout : This parameter specifies the maximum time allowed for an ECU to respond to a diagnostic request. If the response is not received within this timeout period, the diagnostic tool may consider the request as failed.
  4. S3 Timeout : This parameter specifies the time interval for which an ECU should stay in the “response pending” state after receiving a request. If the ECU does not provide a response within this timeout, the diagnostic tool may consider the request as failed.
  5. Communication Timeout : This parameter sets the maximum time allowed for the completion of a diagnostic communication session between the diagnostic tool and the ECU.

Proper configuration and management of timing parameters in UDS are essential for ensuring efficient and reliable diagnostic communication, minimizing delays, and preventing communication errors. Timing parameters help in optimizing the timing aspects of diagnostic services and ensuring timely exchange of diagnostic information between ECUs and diagnostic tools.

Abbreviation

Abbreviation Description
BRS bit rate switch
BS BlockSize
CAN controller area network
CAN FD controller area network with flexible data rate and larger payload as defined in ISO 11898-1
CLASSICAL CAN controller area network with static data rate and up to 8 data bytes as defined in ISO 11898-1
CF ConsecutiveFrame
CTS continue to send
DA destination address
DLC CAN frame data link layer data length code
DoCAN diagnostic communication over controller area network
ECM Engine Control Module
ECU Electronic Control Unit
FC FlowControl
FF FirstFrame
FF_DL FirstFrame data length in bytes
FMI failure mode indicator
FS FlowStatus
ID identifier
Mtype message type
N/A not applicable
NA network address
N_AE network address extension
N_AI network address information
N_Ar network layer timing parameter Ar
N_As network layer timing parameter As
N_Br network layer timing parameter Br
N_Bs network layer timing parameter Bs
N_ChangeParameter network layer service name
N_Cr network layer timing parameter Cr
N_Cs network layer timing parameter Cs
N_Data network data
N_PCI network protocol control information
N_PCItype network protocol control information type
N_PDU network protocol data unit
N_SA network source address
N_SDU network service data unit
N_TA network target address
SOM start of message
SP nominal sample point
SPN suspect parameter number
STmin Separation Time minimum
STRT serviceToRespondTo
TA target address
TCM transmission control module
UDS unified diagnostic services
USDT unacknowledged segmented data transfer
UUDT unacknowledged unsegmented data transfer
WWH-OBD world-wide harmonized on-board diagnostics
ISO 15031 OBD
ISO 15765 DoCAN
ISO 13400 DoIP
ISO 14229 UDS
ISO 27145 WWH-OBD

FreeRTOS: An Introduction and Porting Guide for STM32F429

Introduction to FreeRTOS

FreeRTOS is a popular real-time operating system (RTOS) designed for embedded systems, providing a robust and lightweight solution for developing complex, time-critical applications. Developed by Real Time Engineers Ltd., FreeRTOS has become a go-to choice for microcontroller-based projects across various industries, including automotive, industrial automation, and consumer electronics.

What is an RTOS?

A Real-Time Operating System (RTOS) is specifically designed to handle time-sensitive tasks with predictable and deterministic timing. Unlike general-purpose operating systems, an RTOS ensures that critical tasks are completed within strict time constraints, making it essential for applications where timing is crucial.

Core Features of FreeRTOS

1. Task Management

FreeRTOS implements a priority-based preemptive scheduling mechanism. Key characteristics include:

  • Flexible task creation and management
  • Support for multiple task priorities
  • Dynamic task creation and deletion
  • Lightweight task switching mechanism

2. Synchronization Primitives

The operating system provides robust synchronization mechanisms:

  • Mutexes for resource protection
  • Semaphores for task synchronization
  • Queues for inter-task communication
  • Event groups for complex synchronization scenarios

3. Memory Management

FreeRTOS offers multiple memory allocation strategies:

  • Static memory allocation
  • Dynamic memory allocation
  • Heap memory management with configurable allocation schemes

4. Low Resource Footprint

Designed for resource-constrained environments, FreeRTOS:

  • Requires minimal RAM and ROM
  • Offers configurable kernel features
  • Supports a wide range of microcontrollers and architectures

Porting FreeRTOS to STM32F429

Preparation

Before porting FreeRTOS to STM32F429, ensure you have:

  • STM32F429 development board
  • MDK-ARM (Keil uVision) or equivalent development environment
  • FreeRTOS source code
  • STM32F429 hardware reference manual

Step-by-Step Porting Process

1. Project Setup

  1. Create a new MDK-ARM project for STM32F429
  2. Include FreeRTOS source files in your project
  3. Configure project include paths

2. Port-Specific Configuration

Modify FreeRTOSConfig.h to customize the RTOS for STM32F429:

1
2
3
#define configCPU_CLOCK_HZ    ( SystemCoreClock )
#define configTICK_RATE_HZ ( ( TickType_t ) 1000 )
#define configTOTAL_HEAP_SIZE ( ( size_t ) ( 64 * 1024 ) )

3. Implement Processor-Specific Ports

Create port files specific to ARM Cortex-M4:

  • port.c: Contains context switching and interrupt handling
  • portmacro.h: Defines processor-specific macros

4. Interrupt Management

Configure system timer and PendSV interrupt for task switching:

1
2
3
4
5
6
7
8
9
void SysTick_Handler(void) {
#if (INCLUDE_xTaskGetSchedulerState == 1)
if (xTaskGetSchedulerState() != taskSCHEDULER_NOT_STARTED) {
#endif
xPortSysTickHandler();
#if (INCLUDE_xTaskGetSchedulerState == 1)
}
#endif
}

5. Basic Task Creation Example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
void vTaskMain(void *pvParameters) {
for (;;) {
// Task implementation
vTaskDelay(pdMS_TO_TICKS(1000));
}
}

int main(void) {
// Hardware initialization

xTaskCreate(vTaskMain,
"MainTask",
configMINIMAL_STACK_SIZE,
NULL,
tskIDLE_PRIORITY,
NULL);

vTaskStartScheduler();

// Should never reach here
for (;;);
}

Common Challenges and Solutions

  1. Interrupt Handling: Carefully manage interrupt priorities and nesting
  2. Memory Constraints: Optimize memory usage through static allocation
  3. Timing Precision: Calibrate system tick and task scheduling

Best Practices

  • Use static allocation when possible
  • Implement proper task priority management
  • Utilize built-in synchronization primitives
  • Regularly profile and optimize task performance
  • Implement error handling and monitoring mechanisms

Conclusion


Porting FreeRTOS to STM32F429 provides a powerful framework for developing real-time embedded applications. By understanding the core features and following a systematic porting approach, developers can leverage the full potential of both the microcontroller and the real-time operating system.

Reference Implementation Details

Key Code Snippets from apexpeng/FreeRTOS_STM32F429

1. System Initialization (main.c)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
int main(void)
{
// System Clock Configuration
SystemInit();

// NVIC Priority Grouping Configuration
NVIC_PriorityGroupConfig(NVIC_PriorityGroup_4);

// LED Initialization
LED_Init();

// USART Initialization for Debugging
USART1_Init();

// Create Tasks
xTaskCreate(LED0_Task, "LED0 Task", 50, NULL, 1, NULL);
xTaskCreate(LED1_Task, "LED1 Task", 50, NULL, 2, NULL);

// Start FreeRTOS Scheduler
vTaskStartScheduler();

// Should never reach here
while(1);
}

2. Task Implementation Example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
void LED0_Task(void *pvParameters)
{
while(1)
{
LED0_Toggle();
vTaskDelay(500); // Delay for 500 milliseconds
}
}

void LED1_Task(void *pvParameters)
{
while(1)
{
LED1_Toggle();
vTaskDelay(800); // Delay for 800 milliseconds
}
}

3. FreeRTOS Configuration (FreeRTOSConfig.h)

1
2
3
4
5
6
7
8
#define configUSE_PREEMPTION              1
#define configUSE_PORT_OPTIMISED_TASK_SELECTION 1
#define configUSE_TICKLESS_IDLE 0
#define configCPU_CLOCK_HZ ( SystemCoreClock )
#define configTICK_RATE_HZ ( ( TickType_t ) 1000 )
#define configMAX_PRIORITIES ( 5 )
#define configMINIMAL_STACK_SIZE ( ( uint16_t ) 128 )
#define configTOTAL_HEAP_SIZE ( ( size_t ) ( 16 * 1024 ) )

4. Interrupt Handling Example

1
2
3
4
5
6
7
8
9
void USART1_IRQHandler(void)
{
if(USART_GetITStatus(USART1, USART_IT_RXNE) != RESET)
{
uint16_t data = USART_ReceiveData(USART1);
// Handle received data
USART_ClearITPendingBit(USART1, USART_IT_RXNE);
}
}

Practical Considerations

The reference implementation demonstrates several critical aspects of FreeRTOS integration:

  • Proper system initialization
  • Task creation with different priorities
  • Simple task implementations
  • Interrupt handling
  • Minimal configuration for STM32F429

References

NOTE: The DAC function only works fine before tasks scheduler, the clock need to be set to fix this issue.