Ship Sensor Data Platform for Navy Digital
Client Background
The Royal Navy, a vital branch of the UK Ministry of Defence, operates a diverse fleet of ships across the globe. The development of applications demanded real-time access to accurate sensor data to ensure crew safety, optimise resource allocation, and maintain a high level of situational awareness. However, their existing data infrastructure in some areas was siloed, outdated, and unable to meet these modern demands.
Challenge
Faced with these limitations, Navy Digital, the digital transformation arm of the Ministry of Defence, proposed a modern data platform to capture, process, and expose ship sensor data. This platform needed to provide real-time insights to applications used by the crew, enabling them to make informed decisions and enhance operational efficiency.
Our Mission
As architectural lead at Navy Digital, I led the design and implementation of a new data platform, focusing on:
- Ingesting sensor data:
Architecting a scalable data pipeline using Apache Nifi and Kafka on Kubernetes to capture and stream sensor data from diverse sources in real-time. - Stream processing and enrichment:
Designing data processing pipelines to cleanse, transform, and enrich sensor data using stream processing frameworks, extracting valuable insights for applications. - GraphQL API exposure:
Defining and implementing a GraphQL API using open-source technologies, providing a flexible and intuitive interface for crew applications to access and utilise sensor data.
Impact
The new data platform would deliver significant benefits:
- Enhanced situational awareness: Real-time sensor data empowered crew with clear insights into ship operations and the surrounding environment, improving decision-making capabilities.
- Increased operational efficiency: Streamlined data access and analysis enabled faster response times and optimised resource allocation, leading to improved mission effectiveness.
- Modern and scalable architecture: The Kubernetes-based infrastructure provided scalability and flexibility to accommodate future data growth and changing requirements.
Expertise applied
This project wasn't just about technology. We actively collaborated with stakeholders across the organisation:
- Stakeholder engagement: We actively liaised with senior Navy stakeholders, translating their needs into technical requirements and ensuring solution alignment with strategic goals.
- Design leadership: We owned the high-level design, facilitated design reviews, and fostered collaboration among architects and engineers, ensuring a unified and high-quality solution.
- Project delivery: We played a key role in leading the project to successful implementation, overcoming challenges and ensuring timely delivery.
- Solution architecture: Designing and implementing complex distributed data platforms for real-time data pipelines and API exposure.
- Technology selection and integration: Choosing and integrating cutting-edge open-source technologies like Apache Kafka, Kubernetes, and GraphQL.
