Streaming Data Processing & Analytics on Water Supply Systems

The operation and management of water supply systems (WSS) have become increasingly difficult and complex. WSSs currently collect a large amount of data signals (e.g., pressure, flow) at different time frequencies in order to obtain useful information about the current and future state of the system. However, there are several knowledge gaps and challenges related to data flow, including: i) poor data quality and lack of tools to process and integrate data from different sources; and ii) the effective use of a huge amount of collected and stored data. These gaps highlight the need to improve data governance practices and use the collected data more effectively to increase the resilience and efficiency of the WSS.

The main objective of the StreamWater project is the efficient processing of real-time data from large water data streams to improve daily operation and management. The data streams will occur with different frequencies and synchronization intervals. In addition, it also aims to create more accurate data-based techniques for the detection and real-time location of anomalous events in the SAA. Finally, a roadmap will be developed with the lessons learned during the project, allowing other national and international water supply companies to replicate and improve the methodology for processing and analyzing the water data stream.

The proposed methodology for this one-year research project includes three tasks, focusing on the case study of a Portuguese water supply company, Inframoura, namely:

• T1 - Continuous flow data processing;
• T2 - Detection and location of anomalous events in real time;
• T3 - Implementation of the prototype and development of a roadmap.

Inframoura manages the water supply of the coastal city of Vilamoura, one of the largest tourist complexes in Portugal, with several facilities, 22,000 inhabitants and a high seasonal variation in demand (5x higher in summer). The monitoring system consists of 24 flow meters, 20 pressure sensors and about 17,000 smart meters installed at customers’ premises (i.e., households).

This project will be the continuation of three research projects funded by the FCT, two coordinated by the principal investigator (Nelson Carriço) and one by Dídia Covas, including research applied to SAA challenges: DECIdE (2017/2020) aimed to develop a platform to integrate data from different information systems (IS) for the calculation of performance indicators and water and energy balances; WISDom (2019/2022) with the aim of developing algorithms and models to extract knowledge from data, supporting water companies in decision making; and IMiST (2018/2022) focused on improving mixing conditions in water storage reservoirs.

The R&D research team has extensive experience in numerical modeling of WSS dealing with multiple problems (e.g., leak detection, sensor location, optimizations), as well as in experimental analysis and full-scale testing. The team includes different skills, including civil engineers, mathematicians, data science managers and computer engineers.

The expected results of the project include a roadmap with a set of guidelines for the processing of water supply data streams for small and medium-sized management entities and a functional prototype for the processing and analysis of real-time water consumption data in the Inframoura WSS. These results will increase the quality of the data and the use of the data streams, which, consequently, will increase the efficiency of the management and operation of the water supply systems, and can be extended to other systems both nationally and internationally.