Company: CIC Consulting Informáticos
Development of drivers and real-time streaming systems with Kafka for energy data monitoring at CIC.
Backend Developer & DevOps Engineer - IDboxRT Platform
November 2021 - Present (3+ years)
Specialized backend development and DevOps engineering on the IDboxRT industrial IoT monitoring platform. Leading implementation of distributed architectures with advanced streaming technologies (Kafka, Apache Storm) and managing production cloud infrastructure on OVH.
Working on IDboxRT - Enterprise industrial IoT drivers and real-time monitoring solution.
Full-stack platform development
This role encompasses the complete spectrum from industrial IoT driver development in C# and Java to DevOps infrastructure management, providing deep expertise in industrial data processing and cloud operations.
Industrial IoT Driver Development
Developed custom industrial IoT drivers and tasks for the IDboxRT platform:
Driver Development (C# & Java)
- Protocol implementations for industrial equipment communication
- Data acquisition from sensors, PLCs, and industrial devices
- Custom tasks for data transformation and enrichment
- Error handling and retry logic for unreliable industrial networks
Integration Ecosystem
- Support for multiple industrial protocols (Modbus, OPC UA, MQTT)
- Real-time data collection from diverse equipment
- Edge computing capabilities for local processing
Industrial impact
These drivers enable IDboxRT to monitor and collect data from thousands of industrial devices across energy facilities, manufacturing plants, and critical infrastructure.
Distributed Microservices Architecture
Designed and implemented a complete distributed stream processing architecture using cutting-edge technologies:
Technology Stack
Apache Kafka - Event Streaming Backbone
- Primary data ingestion pipeline for all industrial events
- Topic partitioning for parallel processing
- Data retention policies for compliance and analysis
Zookeeper - Distributed Coordination
- Service discovery for dynamic microservice registration
- Configuration management across distributed systems
- Leader election for fault-tolerant coordination
Apache Storm - Real-Time Processing
- Stream processing of industrial data
- Complex event processing (CEP) for anomaly detection
- Real-time aggregations and computations
Performance Characteristics
| Metric | Capability |
|---|---|
| Throughput | Thousands of events per second |
| Latency | Sub-second processing |
| Scalability | Horizontal scaling across multiple nodes |
| Reliability | Fault-tolerant with automatic recovery |
Enterprise-grade streaming
This architecture processes massive volumes of industrial data in real-time, enabling immediate insights for energy management and operational optimization.
OVH Cloud Infrastructure Management
Complete ownership of production infrastructure on OVH Cloud for the IDboxRT platform:
Infrastructure Responsibilities
| Area | Implementation | Impact |
|---|---|---|
| Security | Firewall rules, network segmentation, VPN access | Secured production environment |
| Networking | Reverse proxies, load balancers, DNS management | High availability and performance |
| Containers | Docker orchestration and deployment | Consistent, reproducible deployments |
| Monitoring | 24/7 system monitoring with Zabbix | Proactive issue detection |
| Automation | CI/CD pipelines, deployment scripts | Reduced manual operations |
Database Optimization
MongoDB - NoSQL for Industrial Data
- Advanced indexing strategies for query performance
- Sharding configuration for horizontal scaling
- Replication for data redundancy and read scaling
- Optimized for time-series industrial data storage
Redis - Caching & Real-Time Data
- Caching layer for frequently accessed data
- Pub/Sub for real-time event distribution
- Session management for distributed systems
- Performance tuning for sub-millisecond latency
High Availability Architecture
- Load balancing across multiple application instances
- Database replication with automatic failover
- Backup automation with disaster recovery procedures
- 99.9% uptime maintained across production environments
Production responsibility
Managing production infrastructure for industrial monitoring means ensuring 24/7 availability - any downtime directly impacts energy facilities and manufacturing operations.
Advanced Monitoring with Zabbix
Implemented enterprise-grade monitoring for complex distributed systems:
Zabbix Monitoring Implementation
Automated Alerting System
- Multi-channel notifications - SMS, Email, Slack integration
- Smart escalation policies - Based on severity and business hours
- Incident correlation - Grouping related alerts to reduce noise
Custom Dashboards
- Real-time metrics visualization for system health
- Performance trending for capacity planning
- Business KPIs tracking (events processed, system latency)
Comprehensive Monitoring Coverage
- Infrastructure metrics - CPU, memory, disk, network
- Application metrics - Request rates, error rates, latency
- Database performance - Query performance, connection pools
- Kafka cluster health - Topic lag, broker status, consumer groups
- Custom industrial metrics - Device connectivity, data quality
Monitoring Impact
| Benefit | Achievement |
|---|---|
| MTTR (Mean Time To Recovery) | Reduced by 70% with proactive alerting |
| Incident detection | From manual discovery to automated detection in <2 minutes |
| System visibility | 100% coverage of critical components |
| Preventive maintenance | Trend analysis preventing 80% of potential outages |
Proactive operations
The advanced monitoring system transformed operations from reactive firefighting to proactive issue prevention and capacity planning.
Key Achievements & Results
Technical Accomplishments
- Designed distributed architecture processing thousands of industrial events per second
- Developed C# and Java drivers for industrial equipment integration
- Built streaming pipeline with Kafka, Zookeeper, and Apache Storm
- Optimized databases (MongoDB, Redis) for industrial data patterns
- Managed production infrastructure on OVH Cloud with 99.9% uptime
Infrastructure Evolution
Before:
- Limited scalability
- Manual monitoring and alerting
- Reactive incident response
- Inconsistent deployment processes
After:
- Horizontally scalable distributed architecture
- Automated monitoring with proactive alerting
- Predictive maintenance preventing outages
- Fully automated deployment pipelines
Business Impact
- Platform reliability - 99.9% uptime for industrial monitoring clients
- Data processing capacity - Scaled to handle enterprise-level data volumes
- Operational efficiency - Reduced incident response time by 70%
- Development velocity - Faster feature delivery through automation
Complete Technology Stack
Backend Development
- Languages: C#, Java, Python
- Frameworks: Spring Boot, .NET Core
- Messaging: Apache Kafka, RabbitMQ
- Streaming: Apache Storm, Kafka Streams
Databases & Storage
- NoSQL: MongoDB (sharded clusters)
- Cache: Redis (cluster mode)
- Time-Series: Optimized MongoDB collections
DevOps & Cloud
- Cloud Provider: OVH Cloud
- Containerization: Docker
- Monitoring: Zabbix (comprehensive monitoring)
- CI/CD: Custom automation scripts
- Networking: Nginx, HAProxy, VPNs
Distributed Systems
- Coordination: Apache Zookeeper
- Service Discovery: Custom implementation
- Load Balancing: HAProxy, Nginx
- Message Queuing: Kafka, RabbitMQ
Ongoing role
This role continues alongside newer responsibilities, demonstrating the ability to maintain and evolve existing systems while taking on new challenges like the Rabel platform development.
