EdgeSense
Active ProjectHardware
A comprehensive IoT sensor integration system for EdgeSense, a native macOS AI desktop application. The update enables real-time data streaming from edge devices—including Raspberry Pi, Arduino (ESP32/ESP8266), and Nvidia Orin Nano—directly into LLM conversations. By leveraging WebSocket communication, mDNS zero-configuration discovery, and an efficient in-memory ring buffer architecture, the system provides seamless contextual awareness to AI assistants about physical world conditions.
4
Total Feedback
Feb 28
Last Updated
Key Features
Remote DevicesEdge DevicesMacOS AppClient App+7 more
Tech Stack
RustTauri 2.0Axumtokio+8 more
Image Gallery
EdgeSense Project
Loading updates...
Project Roadmap
Loading timeline...
Upcoming Features
As a user, I want to be able to track and revert changes to tool scripts in EdgeSense because this will enable version control, allowing me to view the history of script changes and restore previous versions without data loss. This will affect the Version History section, where I will interact with a new UI component to manage script versions, and the tool script management pages, where I will see options to save, list, and revert versions.Planned
Medium Priority
As a user, I want to be able to export and import portable tool and app bundles between EdgeSense installations because this will allow me to seamlessly transfer my configurations, custom models, and user data without data loss or corruption. This will affect the settings or tools menu, where I will be able to access new options for managing export/import interactions through a dialog component.Planned
Medium Priority
As a user, I want to be able to schedule my tools in EdgeSense to run automatically at specified intervals using cron expressions because this will allow me to automate repetitive tasks without needing manual intervention. This will affect the Schedule Manager section, where I will be able to input cron expressions and manage my tool schedules. The changes will also be reflected in the tool management interface, enabling me to link schedules with specific tools.Planned
Medium Priority
As a user, I want to be able to collaborate on a network map in real-time with my team because it will enable us to share map updates instantly across multiple EdgeSense instances without data loss or delays. This will affect the MapComponent section, where I will see real-time updates, and the underlying state management, ensuring that all users view the same synchronized map data.Planned
Medium Priority
Known Issues
No known issues
Project Challenges
1.Cross-Platform mDNS Compatibility
Challenge: mDNS behaves differently across macOS, Windows, and Linux. The
Solution: Implemented explicit interface selection prioritising non-loopback IPv4 addresses using the
2. WebSocket Connection Stability on Embedded Devices
Challenge: ESP8266/ESP32 devices have limited memory and unstable WiFi connections, causing frequent disconnections that weren't always detected promptly.
Solution: Implemented heartbeat mechanism with configurable timeout, combined with exponential backoff reconnection.
3. Efficient State Synchronisation Between Rust and React
Challenge: Tauri's IPC is efficient but batching many small sensor updates created UI lag.
Solution: Used Tokio broadcast channels in Rust and Tauri's event system for push-based updates, rather than polling.
4. Token Budget Management for LLM Context
Challenge: Sensor context can grow unboundedly with many devices, potentially consuming the LLM's context window.
Solution: Implemented configurable token budget with intelligent truncation.
``
5. Ring Buffer Memory Estimation
Challenge: Estimating appropriate buffer sizes for diverse sensor reading frequencies without excessive memory consumption.
Solution: Settled on 1000 readings per sensor type as default, with configurable limits. At ~200 bytes per reading, this bounds memory to ~200KB per sensor type per device—acceptable for desktop use.
Challenge: mDNS behaves differently across macOS, Windows, and Linux. The
mdns-sd crate abstracts much of this, but network interface enumeration proved problematic on machines with multiple interfaces (Ethernet + WiFi + VPN).Solution: Implemented explicit interface selection prioritising non-loopback IPv4 addresses using the
getifaddrs crate.2. WebSocket Connection Stability on Embedded Devices
Challenge: ESP8266/ESP32 devices have limited memory and unstable WiFi connections, causing frequent disconnections that weren't always detected promptly.
Solution: Implemented heartbeat mechanism with configurable timeout, combined with exponential backoff reconnection.
3. Efficient State Synchronisation Between Rust and React
Challenge: Tauri's IPC is efficient but batching many small sensor updates created UI lag.
Solution: Used Tokio broadcast channels in Rust and Tauri's event system for push-based updates, rather than polling.
4. Token Budget Management for LLM Context
Challenge: Sensor context can grow unboundedly with many devices, potentially consuming the LLM's context window.
Solution: Implemented configurable token budget with intelligent truncation.
``
rust
let maxchars = self.config.maxtokens * 4; // ~4 chars per token
if context.len() > maxchars {
context.truncate(maxchars - 20);
context.push_str("\n... [truncated]");
}
``5. Ring Buffer Memory Estimation
Challenge: Estimating appropriate buffer sizes for diverse sensor reading frequencies without excessive memory consumption.
Solution: Settled on 1000 readings per sensor type as default, with configurable limits. At ~200 bytes per reading, this bounds memory to ~200KB per sensor type per device—acceptable for desktop use.
Project Solutions & Learnings
1.Rust's Ownership Model Shines for Concurrent State
Managing concurrent device connections would be error-prone in C++ or prone to race conditions in JavaScript. Rust's
2. mDNS is Powerful but Fragile
Zero-configuration discovery significantly improves user experience for IoT integrations, but mDNS has quirks:
- Some routers filter multicast traffic
- Virtual networks (Docker, VMs) may not propagate mDNS
- Resolution can be slow (2-5 seconds)
Always provide manual host configuration as a fallback.
3. Keep the Wire Protocol Simple
JSON over WebSocket may not be the most efficient choice, but its debuggability is invaluable. During development, being able to
4. Design for Graceful Degradation
The system continues functioning when:
- No devices are connected (empty context)
- mDNS fails (manual host fallback)
- Device disconnects mid-stream (heartbeat timeout cleanup)
- Rate limits are exceeded (device informed, server protected)
This resilience is essential for production IoT systems where reliability trumps feature completeness.
5. Frontend State Management Complexity
Real-time data introduces complexity that simpler CRUD applications avoid:
- Optimistic updates vs. server truth
- Stale data detection
- Event ordering guarantees
Zustand with persist middleware handled this well, but required careful thought about which state should persist (configuration) versus which should be ephemeral (device connections).
Managing concurrent device connections would be error-prone in C++ or prone to race conditions in JavaScript. Rust's
Arc> pattern with parking_lot's implementation provided both safety and performance. The compiler catches potential data races at compile time, which proved invaluable during development.2. mDNS is Powerful but Fragile
Zero-configuration discovery significantly improves user experience for IoT integrations, but mDNS has quirks:
- Some routers filter multicast traffic
- Virtual networks (Docker, VMs) may not propagate mDNS
- Resolution can be slow (2-5 seconds)
Always provide manual host configuration as a fallback.
3. Keep the Wire Protocol Simple
JSON over WebSocket may not be the most efficient choice, but its debuggability is invaluable. During development, being able to
wscat into the server and send hand-crafted messages accelerated debugging significantly. For IoT devices with limited bandwidth, binary protocols (MessagePack, Protobuf) could be future optimisations.4. Design for Graceful Degradation
The system continues functioning when:
- No devices are connected (empty context)
- mDNS fails (manual host fallback)
- Device disconnects mid-stream (heartbeat timeout cleanup)
- Rate limits are exceeded (device informed, server protected)
This resilience is essential for production IoT systems where reliability trumps feature completeness.
5. Frontend State Management Complexity
Real-time data introduces complexity that simpler CRUD applications avoid:
- Optimistic updates vs. server truth
- Stale data detection
- Event ordering guarantees
Zustand with persist middleware handled this well, but required careful thought about which state should persist (configuration) versus which should be ephemeral (device connections).