The Development of Real-Time Fouling Monitoring and Control Systems for Reverse Osmosis Membrane Cleaning using Deep Reinforcement Learning

Abstract

This dissertation investigates potential applications for Machine Learning (ML) and real-time fouling monitors in Reverse Osmosis (RO) desalination. The main objective was to develop a framework that minimizes the cost of membrane fouling by deploying AI-generated cleaning patterns and real-time fouling monitoring. Membrane manufacturers and researchers typically recommend cleaning (standard operating procedure – SOP) when normalized permeate flow, a performance metric tracking the decline of permeate flow/output from its initial baseline with respect to operating pressure, reaches 0.85-0.90 of baseline values. This study used estimates of production cost, internal profitability metrics, and permeate volume output to evaluate and compare the impact of time selection for cleaning intervention. The cleanings initiated when the normalized permeate flow reached 0.85 represented the control for cleaning intervention times. In deciding optimal times for cleaning intervention, a Deep Reinforcement Learning (RL) agent was trained to signal cleaning between 0.85-0.90 normalized with a cost-based reward system. A laboratory-scale RO flat membrane desalination system platform was developed as a model plant, and data from the platform and used to train the model and examine both simulated and actual control of when to trigger membrane cleaning, replacing the control operator's 0.85 cleaning threshold. Compared to SOP, the intelligent operator showed consistent savings in production costs at the expense of total permeate volume output. The simulated operation using the RL initiated yielded 9% less permeate water but reduced the cost per unit volume (/m3)by12.3/m3) by 16.0%. These results are consistent with an RL agent that prioritizes production cost savings over product volume output.