EdgeSense
Active ProjectHardware
A comprehensive IoT sensor integration system for EdgeSense, a native macOS AI desktop application. The update enables real-time data streaming from edge devices—including Raspberry Pi, Arduino (ESP32/ESP8266), and Nvidia Orin Nano—directly into LLM conversations. By leveraging WebSocket communication, mDNS zero-configuration discovery, and an efficient in-memory ring buffer architecture, the system provides seamless contextual awareness to AI assistants about physical world conditions.
19
Total Feedback
May 28
Last Updated
Key Features
Remote DevicesEdge DevicesMacOS AppClient App+7 more
Tech Stack
RustTauri 2.0Axumtokio+8 more
Image Gallery
v1.39 - MCP: Sensor Summary
v1.39 - MCP: Sensor Summary
v1.39 - MCP: App Creation (Example)
v1.39 - MCP: App Creation
EdgeSense - LLM Chat Window
EdgeSense - Local and API Model Configuration
EdgeSense - Available Devices & Sensors
EdgeSense - Edit Edge Device Sensor Config
EdgeSense - Saved Tools
EdgeSense - Saved Apps
EdgeSense - Connect App Page
EdgeSense - Split View w/Network
EdgeSense - Network View
EdgeSense - Settings Page
EdgeSense - MCP Server Configuration
EdgeSense - Client Install Guide
EdgeSense - App Example
EdgeSense Project
Loading updates...
Project Roadmap
Loading timeline...
Upcoming Features
Add a visual indication to the chat section where chats contain cron jobs (Tools). Add a section to the chat window summary called 'Notifications' that shows all of the last responses from chats with cron automations in them - this page should effectivey show all of the users automations (From tools) and what their latest responses are.Planned
Medium Priority
Enable a folder structure within the chat window on the left hand section so the user can organise chats more effectively.Planned
Medium Priority
Enable running of EdgeSense on edge devices like the Orin Nano.Planned
Medium Priority
Enable the creation of CRON jobs as tools so that the LLM can be used to perform repeated tasks (Short (every second) and Long term (every hour)) to provide continual updates to the user using the tools in the app.Planned
Medium Priority
Enable the creation of CRON jobs as tools so that the LLM can be used to perform repeated tasks (Short (every second) and Long term (every hour)) to provide continual updates to the user using the tools in the app.Planned
Medium Priority
Known Issues
No known issues
Downloads
EdgeSense v1.39.0 - MacOS (Apple Silicon)
DMG163.4 MB
Project Challenges
1.Cross-Platform mDNS Compatibility
Challenge: mDNS behaves differently across macOS, Windows, and Linux. The
Solution: Implemented explicit interface selection prioritising non-loopback IPv4 addresses using the
2. WebSocket Connection Stability on Embedded Devices
Challenge: ESP8266/ESP32 devices have limited memory and unstable WiFi connections, causing frequent disconnections that weren't always detected promptly.
Solution: Implemented heartbeat mechanism with configurable timeout, combined with exponential backoff reconnection.
3. Efficient State Synchronisation Between Rust and React
Challenge: Tauri's IPC is efficient but batching many small sensor updates created UI lag.
Solution: Used Tokio broadcast channels in Rust and Tauri's event system for push-based updates, rather than polling.
4. Token Budget Management for LLM Context
Challenge: Sensor context can grow unboundedly with many devices, potentially consuming the LLM's context window.
Solution: Implemented configurable token budget with intelligent truncation.
``
5. Ring Buffer Memory Estimation
Challenge: Estimating appropriate buffer sizes for diverse sensor reading frequencies without excessive memory consumption.
Solution: Settled on 1000 readings per sensor type as default, with configurable limits. At ~200 bytes per reading, this bounds memory to ~200KB per sensor type per device—acceptable for desktop use.
Challenge: mDNS behaves differently across macOS, Windows, and Linux. The
mdns-sd crate abstracts much of this, but network interface enumeration proved problematic on machines with multiple interfaces (Ethernet + WiFi + VPN).Solution: Implemented explicit interface selection prioritising non-loopback IPv4 addresses using the
getifaddrs crate.2. WebSocket Connection Stability on Embedded Devices
Challenge: ESP8266/ESP32 devices have limited memory and unstable WiFi connections, causing frequent disconnections that weren't always detected promptly.
Solution: Implemented heartbeat mechanism with configurable timeout, combined with exponential backoff reconnection.
3. Efficient State Synchronisation Between Rust and React
Challenge: Tauri's IPC is efficient but batching many small sensor updates created UI lag.
Solution: Used Tokio broadcast channels in Rust and Tauri's event system for push-based updates, rather than polling.
4. Token Budget Management for LLM Context
Challenge: Sensor context can grow unboundedly with many devices, potentially consuming the LLM's context window.
Solution: Implemented configurable token budget with intelligent truncation.
``
rust
let maxchars = self.config.maxtokens * 4; // ~4 chars per token
if context.len() > maxchars {
context.truncate(maxchars - 20);
context.push_str("\n... [truncated]");
}
``5. Ring Buffer Memory Estimation
Challenge: Estimating appropriate buffer sizes for diverse sensor reading frequencies without excessive memory consumption.
Solution: Settled on 1000 readings per sensor type as default, with configurable limits. At ~200 bytes per reading, this bounds memory to ~200KB per sensor type per device—acceptable for desktop use.
Project Solutions & Learnings
1.Rust's Ownership Model Shines for Concurrent State
Managing concurrent device connections would be error-prone in C++ or prone to race conditions in JavaScript. Rust's
2. mDNS is Powerful but Fragile
Zero-configuration discovery significantly improves user experience for IoT integrations, but mDNS has quirks:
- Some routers filter multicast traffic
- Virtual networks (Docker, VMs) may not propagate mDNS
- Resolution can be slow (2-5 seconds)
Always provide manual host configuration as a fallback.
3. Keep the Wire Protocol Simple
JSON over WebSocket may not be the most efficient choice, but its debuggability is invaluable. During development, being able to
4. Design for Graceful Degradation
The system continues functioning when:
- No devices are connected (empty context)
- mDNS fails (manual host fallback)
- Device disconnects mid-stream (heartbeat timeout cleanup)
- Rate limits are exceeded (device informed, server protected)
This resilience is essential for production IoT systems where reliability trumps feature completeness.
5. Frontend State Management Complexity
Real-time data introduces complexity that simpler CRUD applications avoid:
- Optimistic updates vs. server truth
- Stale data detection
- Event ordering guarantees
Zustand with persist middleware handled this well, but required careful thought about which state should persist (configuration) versus which should be ephemeral (device connections).
Managing concurrent device connections would be error-prone in C++ or prone to race conditions in JavaScript. Rust's
Arc> pattern with parking_lot's implementation provided both safety and performance. The compiler catches potential data races at compile time, which proved invaluable during development.2. mDNS is Powerful but Fragile
Zero-configuration discovery significantly improves user experience for IoT integrations, but mDNS has quirks:
- Some routers filter multicast traffic
- Virtual networks (Docker, VMs) may not propagate mDNS
- Resolution can be slow (2-5 seconds)
Always provide manual host configuration as a fallback.
3. Keep the Wire Protocol Simple
JSON over WebSocket may not be the most efficient choice, but its debuggability is invaluable. During development, being able to
wscat into the server and send hand-crafted messages accelerated debugging significantly. For IoT devices with limited bandwidth, binary protocols (MessagePack, Protobuf) could be future optimisations.4. Design for Graceful Degradation
The system continues functioning when:
- No devices are connected (empty context)
- mDNS fails (manual host fallback)
- Device disconnects mid-stream (heartbeat timeout cleanup)
- Rate limits are exceeded (device informed, server protected)
This resilience is essential for production IoT systems where reliability trumps feature completeness.
5. Frontend State Management Complexity
Real-time data introduces complexity that simpler CRUD applications avoid:
- Optimistic updates vs. server truth
- Stale data detection
- Event ordering guarantees
Zustand with persist middleware handled this well, but required careful thought about which state should persist (configuration) versus which should be ephemeral (device connections).