inholland/ros2-assignments
3
1
Fork 0
generated from wessel/boilerplate
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[PR] Implement integration test documentation, add mosquitto to dockerfile #11

Merged
vincent merged 5 commits from 2-imu-reader/integration-test-documentation into 2-imu-reader/master 2025-11-05 12:06:39 +01:00
Conversation 0 Commits 5 Files Changed 5 +406 -66
5 changed files with 406 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files

77
doc/architecture/IMU/ESP32-IMU.md Normal file
View File

@@ -0,0 +1,77 @@
# IMU (MPU6886) ESP-IDF
=====================
Small ESP-IDF project that reads an MPU6886 IMU over I2C and publishes readings over MQTT.
Includes an option to run WiFi in STA (client) or AP (access point) mode.
### Features
- I2C communication functions for MPU6886 (accelerometer, gyro, temp)
- MQTT client using `esp-mqtt`
- WiFi: STA or AP mode selectable at build time
- AP mode defaults to static IP 192.168.10.1/24 (configurable in code)
- MQTT, Serial toggle using the boot button on the esp32
## Quick start
### Prerequisites
- ESP-IDF installed and activated (the repo was developed with ESP-IDF v5.x).
- Toolchain and Python deps per ESP-IDF instructions.
### Build and flash
From the IMU root:
```bash
# configure project options
idf.py menuconfig
# build
idf.py build
# flash and monitor (set your serial port)
idf.py -p /dev/ttyUSB0 flash monitor
```
## Project Configuration (menuconfig)
Open `idf.py menuconfig` and there you will see the `ESP32 IMU Project Configuration` submenu where you can configure specific data for I2C, MQTT and WIFI
### I2C
---
In the `I2C Master configuration` submenu there are three options:
- SCL GPIO Num - Default 21 - GPIO used for I2C SDA
- SDA GPIO Num - Default 22 - GPIO used for I2C SCL
- Master Frequency - Default 100000hz - I2C bus speed
### MQTT
- Broker URI - Default: `"mqtt://192.168.4.2:1883"`
- MQTT Topic - Default: `"esp32/imu"`
### WiFi / network
- Toggle AP mode at build time by enabling `CONFIG_WIFI_AP_MODE` in menuconfig. If enabled the firmware will start as an access point on boot; if disabled the device starts as a station and attempts to connect to the configured SSID using the set Password
- Toggle AP mode swaps the available settings when toggled between AP conf- and station configurations respectively
- AP credentials can be set with `WiFi AP SSID` and `Wifi AP Password`.
- Station credentials can be set with `WiFi SSID` and `WiFi Password`.
### AP IP / DHCP
- The AP is configured by default to use a static IP of `192.168.4.1` with netmask `255.255.255.0` and a DHCP server is started so clients receive addresses on that subnet.
## Notes about units and macros
- The macro `SF_RAD_S` (in `main/mpu6886.h`) is the degrees-to-radians conversion factor: PI/180. Gyroscope readings are converted from degrees/sec to radians/sec using this scale.
- Sensitivity scale constants (e.g. `ACCEL_SO_2G`, `GYRO_SO_250DPS`) are used to convert raw sensor counts to physical units.
## Troubleshooting
- "`MQTT_TOPIC` undefined" at build time: run menuconfig
- "I2C init failed": confirm SDA/SCL pins in menuconfig or wiring.
## Files of interest
- `main/mpu6886.c` - main application, WiFi init, MQTT publish loop
- `main/mpu6886.h` - sensor constants and types

85
doc/architecture/architecture.md
View File

@@ -1,15 +1,14 @@
# TI Minor Grade Generator Design Document
# ROS2 IMU Reader — Design Document
## Project Overview
Giving marks to students cost a lot of time and pain. For this a automatic solution is needed.
Therefore the overall Software architect (Tilmann K.) has designed a solution for the ROS2 architecture.
This projects reads data from a ESP32 communicating with serial or MQTT and writes this into a database for further processing
## System Architecture
### High-Level Architecture
The system consists of multiple ROS2 nodes that communicate through standardized topics and services to process exam data, calculate grades, and persist results. The architecture ensures scalability, maintainability, and fault tolerance.
The system consists of multiple ROS2 nodes that communicate through standardized topics and services to process imu data and store it in a database
### Key Design Principles
@@ -20,18 +19,21 @@ The system consists of multiple ROS2 nodes that communicate through standardized
## System Components
### ESP32-IMU
*For ESP32 specific documentation see [ESP32-IMU.md](IMU/ESP32-IMU.md)*
### Core Nodes
#### 1. FinalGradeDeterminator Node
**Namespace**: `assignments::one::final_grade_determinator`
#### 1. IMU Databse writer Node
**Namespace**: `assignments::two::imu_database_writer`
**Brief Description**: Collects exam results, triggers grade calculation when thresholds are met, and stores final grades.
**Brief Description**: Recieves and stores IMU data.
**Key Features**: Configurable collection thresholds, automatic grade calculation triggering, database persistence
**Key Features**: Database persistence
*For detailed documentation, see: [FinalGradeDeterminator.md](nodes/FinalGradeDeterminator.md)*
*For detailed documentation, see: [IMUDatabaseWriter.md](nodes/IMUDatabaseWriter.md)*
#### 2. GradeCalculator Node
#### 2. LifeCycle Node - NEEDS TO BE EDITED STILL
**Namespace**: `assignments::one::grade_calculator`
**Brief Description**: Provides grade calculation service with business logic including bonus points and grade validation.
@@ -40,33 +42,6 @@ The system consists of multiple ROS2 nodes that communicate through standardized
*For detailed documentation, see: [GradeCalculator.md](nodes/GradeCalculator.md)*
#### 3. ExamResultGenerator Node
**Namespace**: `assignments::one::exam_result_generator`
**Brief Description**: Simulates exam result generation by maintaining a queue of student-course combinations and publishing random grades.
**Key Features**: Database-driven queue management, random grade generation, student enrollment handling
*For detailed documentation, see: [ExamResultGenerator.md](nodes/ExamResultGenerator.md)*
#### 4. RetakeScheduler Node
**Namespace**: `assignments::one::retake_scheduler`
**Brief Description**: Manages retake exam scheduling and coordination for students who need to retake exams.
**Key Features**: Retake request processing, schedule management, action server implementation for retake workflows
*For detailed documentation, see: [RetakeScheduler.md](nodes/RetakeScheduler.md)*
#### 5. RetakeGradeDeterminator Node
**Namespace**: `assignments::one::retake_grade_determinator`
**Brief Description**: Handles grade calculation and processing specifically for retake exams.
**Key Features**: Retake-specific grade processing, integration with main grade system, retake result validation
*For detailed documentation, see: [RetakeGradeDeterminator.md](nodes/RetakeGradeDeterminator.md)*
### Data Management
#### DatabaseManager
@@ -85,42 +60,20 @@ The system consists of multiple ROS2 nodes that communicate through standardized
### Communication Interfaces
#### ROS2 Message and Service Interfaces
**Brief Description**: Custom message types and service definitions for inter-node communication.
**Key Components**: Exam and Student message types, Grade calculation service, Retake action interface, standardized timestamps
*For detailed documentation, see: [interfaces.md](interfaces/interfaces.md)*
#### ROS2 Message Interface
**Brief Description**: This project uses the ROS2 standard IMU sensor message
## System Workflow
### 1. Exam Result Processing
1. **Input**: Exam results are published to the `exam_results` topic
2. **Collection**: FinalGradeDeterminator receives and stores exam results by student-course combination
3. **Monitoring**: System tracks the number of results per student-course pair
4. **Threshold Check**: When `grade_collection_amount` results are collected, proceed to calculation
1. **Input**: IMU data is sent from the ESP32 to the lifecycle node
2. **Collection**: The lifcycle node recieves the data via a serial or MQTT connection
4. **Data passthrough**: When data is recieved it is sent to the database writer
### 2. Grade Calculation Process
### 2. Data Management
1. **Service Request**: FinalGradeDeterminator calls GradeCalculator service
2. **Calculation**: GradeCalculator computes final grade using business logic
3. **Validation**: Result is validated and clamped to acceptable range
4. **Response**: Calculated grade is returned to FinalGradeDeterminator
### 3. Result Management
1. **Database Storage**: Final grade is persisted in the database
2. **Publication**: Student information is published to management topic
3. **Logging**: Process completion is logged for audit purposes
### 4. Retake Processing Workflow
1. **Retake Request**: RetakeScheduler receives retake requests via action interface
2. **Schedule Management**: System schedules retake exams and manages timelines
3. **Retake Execution**: Student completes retake exam, results are processed
4. **Grade Processing**: RetakeGradeDeterminator calculates retake grades with specialized logic
5. **Integration**: Retake results are integrated with main grade system and database
1. **Database Storage**: IMU data is persisted/stored in the database
## Configuration Management

284
doc/tests/IntegrationTests.md Normal file
View File

@@ -0,0 +1,284 @@
# IMU System Integration Tests
This document describes integration tests for the IMU data pipeline (IMU reader ESP32, Lifecycle MITM node, Database writer node).
These tests will verify end-to-end functionality from ESP32 sensor data to database storage.
## Test Overview
The integration tests validate the complete flow through the main components:
1. ESP32 MPU6886 Sensor - Data fetching and transmission (MQTT or Serial)
2. ROS2 Lifecycle Node - Message reception from ESP and forwarding to database writer
3. IMU Database Writer Node - Database storage
## Test Environment Setup
### Prerequisites
- ESP32 with MPU6886 sensor configured and flashed
- PostgreSQL database running (can be ran using dockerfile documented in the [README](/README.md))
- ROS2 workspace built and sourced
- if testing MQTT mode, MQTT broker running
- if testing Serial mode, Serial port access
### Configuration Files
- `src/config.toml` - Database connection parameters
- `IMU/sdkconfig` - Serial/MQTT mode selection and network settings
---
## Integration Test Cases
### Test 1: Database Writer Persistence Verification
> Verify that the Database Writer node correctly saves data to the PostgreSQL database when
> IMU messages are received.
#### Test Setup
1. Start PostgreSQL database (docker compose up)
2. Verify database connection, if incorrect change settings settings in `src/config.toml`
3. Clear or note the current state of the IMU data table
4. Launch the Database Writer node:
```bash
ros2 run g2_2025_imu_reader_pkg g2_2025_imu_database_writer_node
```
#### Test Procedure
1. Query the database for initial row count
```sql
SELECT COUNT(*) FROM imu_data;
```
2. Use ROS2 CLI to publish IMU data
```bash
ros2 topic pub --once /imu_data sensor_msgs/msg/Imu "{
linear_acceleration: {x: 0.5, y: 0.6, z: 9.8},
angular_velocity: {x: 0.1, y: 0.2, z: 0.3}
}"
```
3. Query the database for new entries
```sql
SELECT * FROM imu_data ORDER BY timestamp DESC LIMIT 1;
```
4. Send 10 messages with varying data
```bash
for i in {1..10}; do
ros2 topic pub --once /imu_data sensor_msgs/msg/Imu "{
linear_acceleration: {x: $(echo "scale=2; $i * 0.1" | bc), y: 0.0, z: 9.8},
angular_velocity: {x: 0.0, y: 0.0, z: 0.0}
}"
sleep 0.5
done
```
5. Confirm all 10 messages were persisted
```sql
SELECT COUNT(*) FROM imu_data WHERE timestamp > (NOW() - INTERVAL '1 minute');
```
#### Expected Results
- [ ] Each published message creates one database row
- [ ] Linear acceleration values (x, y, z) match published data
- [ ] Angular velocity values (x, y, z) match published data
- [ ] Timestamps are automatically generated and sequential
- [ ] No data loss occurs
### Test 2: Lifecycle Node Message Forwarding
> Verify that the Lifecycle Node correctly receives IMU data from either Serial or MQTT sources and
> forwards it to the `imu_data` topic for consumption by the Database Writer.
#### Test Setup
1. Ensure ROS2 environment is sourced
2. Launch the Lifecycle Node and Database Writer Node:
```bash
ros2 run g2_2025_imu_reader_pkg g2_2025_lifecycle_node
ros2 run g2_2025_imu_reader_pkg g2_2025_imu_database_writer_node
```
4. Monitor the `imu_data` topic:
```bash
ros2 topic echo /imu_data
```
#### Test Procedure - Serial Mode
1. Configure ESP32 for Serial Output:
- Ensure `CONFIG_ENV_MQTT_ENABLED` is not defined in ESP32 sdkconfig
- Flash ESP32 with serial configuration
2. Connect ESP32 via Serial:
- Connect ESP32 to computer via USB
- Identify serial port (e.g., `/dev/ttyUSB0` on Linux)
- Configure lifecycle node to read from this serial port
3. Verify Data Flow:
- Observe lifecycle node logs for incoming serial data
- Confirm `ros2 topic echo /imu_data` displays messages
- Verify message fields match ESP32 output:
- `linear_acceleration.x/y/z` matches `accel.x/y/z`
- `angular_velocity.x/y/z` matches `gyro.x/y/z`
4. Verify Database storage:
- Check database for new entries
- Confirm values match ESP32 sensor readings
#### Test Procedure - MQTT Mode
1. Start MQTT Broker:
```bash
mosquitto -v
```
2. **Configure ESP32 for MQTT Output**:
- Enable `CONFIG_ENV_MQTT_ENABLED` in ESP32 sdkconfig
- Configure MQTT broker URI (`mqtt://192.168.1.100:1883`)
- Set MQTT topic (`CONFIG_MQTT_TOPIC = "imu/data"`)
- Configure WiFi credentials
- Flash ESP32 with MQTT configuration
3. Verify MQTT Publishing:
- Subscribe to MQTT topic to confirm ESP32 is publishing:
```bash
mosquitto_sub -h localhost -t "imu/data" -v
```
- Expected MQTT payload format:
```json
{"accel":{"x":0.123,"y":0.456,"z":9.800},"gyro":{"x":0.012,"y":0.023,"z":0.034},"Temp":25.50}
```
4. Configure Lifecycle Node for MQTT:
- Set lifecycle node to subscribe to MQTT broker and topic
- Restart lifecycle node with MQTT configuration
5. Verify Data Flow:
- Observe lifecycle node logs for incoming MQTT messages
- Confirm `ros2 topic echo /imu_data` displays messages
6. Verify Database storage:
- Check database for continuously arriving data
- Confirm timestamps are recent and sequential
#### Expected Results
**Serial Mode:**
- [ ] Lifecycle node successfully reads JSON-formatted messages from serial port
- [ ] Messages are parsed and converted to `sensor_msgs/msg/Imu` format
- [ ] All IMU data fields are correctly mapped
- [ ] Messages are published to `/imu_data` topic
- [ ] Database writer receives and persists data
**MQTT Mode:**
- [ ] Lifecycle node successfully subscribes to MQTT broker
- [ ] MQTT messages are received and parsed
- [ ] Messages are converted to `sensor_msgs/msg/Imu` format
- [ ] All IMU data fields are correctly mapped
- [ ] Messages are published to `/imu_data`
- [ ] Database writer receives and persists data
### Test 3: ESP32 Data Format Validation
> Verify that the ESP32 correctly formats and transmits IMU data in both Serial
> and MQTT modes according to the expected JSON schema.
#### Test Setup
1. ESP32 with MPU6886 sensor properly wired and powered
2. IMU sensor calibrated (100 samples for gyro and accel)
3. Serial terminal or MQTT subscriber ready to capture output
#### Data Quality Checks
- [ ] Accelerometer Z-axis reads ~9.8 (m/s)^2 when device is stationary and level
- [ ] Gyroscope values near zero when device is stationary
- [ ] Temperature reading is within expected range
- [ ] No NaN or Inf values in output
- [ ] Calibration offsets are properly applied
---
## End-to-End Integration Test
**Objective**: Validate complete system integration from ESP32 sensor to database persistence.
### Test Setup
1. Clean database state (truncate IMU data table)
2. Start PostgreSQL database
3. Start MQTT broker (for MQTT test variant)
4. Launch all ROS2 nodes:
- Lifecycle node
- Database writer node
5. Power on and connect ESP32
### Test Procedure
1. System Initialization:
- Verify all nodes are running and healthy
- Check lifecycle node is connected to data source (Serial/MQTT)
- Confirm database writer node is subscribed to `/imu_data`
2. Data Flow Verification:
- Let system run for 2 minutes
- Monitor ROS2 topics:
```bash
ros2 topic hz /imu_data
ros2 topic bw /imu_data
```
3. **Database Query**:
```sql
SELECT COUNT(*) FROM imu_data WHERE timestamp > (NOW() - INTERVAL '2 minutes');
SELECT
AVG(linear_accel_z) as avg_accel_z,
AVG(angular_vel_x) as avg_gyro_x,
MIN(timestamp) as first_sample,
MAX(timestamp) as last_sample
FROM imu_data
WHERE timestamp > (NOW() - INTERVAL '2 minutes');
```
4. Physical Movement Test:
- Pick up ESP32 and rotate it
- Observe changes in database values
- Verify accelerometer and gyroscope values change
5. Stress Test:
- Let system run for 30 minutes
- Check for memory leaks or connection drops
- Verify continuous data storage
### Expected Results
- [ ] Data flows from ESP32 -> Lifecycle Node -> Database Writer -> PostgreSQL
- [ ] Publishing rate at `/imu_data`
- [ ] Database receives ~240 rows in 2 minutes
- [ ] Average Z-axis acceleration is ~9.8 (m/s)^2 during stationary periods
- [ ] Physical movements are reflected in database values
- [ ] No data loss over extended operation (30 minutes)
- [ ] All components remain stable without crashes
## Test Execution Checklist
### Pre-Test Verification
- [ ] PostgreSQL database is running and accessible
- [ ] Database schema is created (IMU data table exists)
- [ ] ROS2 workspace is built and sourced
- [ ] ESP32 firmware is flashed with correct configuration
- [ ] MQTT broker is running
- [ ] Serial port permissions are correct
### During Test
- [ ] Monitor node logs for errors or warnings
- [ ] Check ROS2 topic publishing rates
- [ ] Verify database connection remains active
- [ ] Observe IMU data values
### Post-Test Analysis
- [ ] Review test results and logs
- [ ] Document any failures or anomalies
- [ ] Clean up test data if necessary

11
docker-compose.yml
View File

@@ -10,3 +10,14 @@ services:
- POSTGRES_DB=grades
ports:
- "5432:5432"
mosquitto:
image: eclipse-mosquitto
container_name: mosquitto
restart: unless-stopped
ports:
- "1883:1883"
- "9001:9001"
volumes:
- ./mosquitto/config:/mosquitto/config
- ./mosquitto/data:/mosquitto/data
- ./mosquitto/log:/mosquitto/log

15
mosquitto/config/mosquitto.conf Normal file
View File

@@ -0,0 +1,15 @@
# Simple Mosquitto configuration allowing anonymous access (insecure for production)
# pid_file /var/run/mosquitto.pid
# Persistence
#persistence true
#persistence_location /var/lib/mosquitto/
# Logging
# Default MQTT listener
listener 1883
allow_anonymous true
# Include additional config fragments