February 2026 Board of Health Meeting - 2025 Predictive Modeling Program Evaluation Auxiliary Report

Meeting Document Type
Auxiliary Report
2025 Predictive Modeling Program Evaluation

EXECUTIVE SUMMARY

The Windsor-Essex County Health Unit (WECHU) monitors local public beaches weekly, collecting water samples to test E. coli bacteria and protect swimmers from potential water-borne-related illnesses. The samples are then sent to the Public Health Ontario laboratory to determine the levels of E. coli bacteria present and the results are provided within 24-48 hours. Upon receiving the results, the WECHU issues water quality warnings when results are between 201-999 E. coli/100ml, and beach closures are issued when results are equal to or greater than 1000 E. coli/100ml. Recognizing that water quality can change from day to day or even hour by hour, some health units including the WECHU are adopting the use of predictive models. These models use current weather conditions and environmental factors to provide timely weekday results for water quality so beach users can make an informed decision before visiting the beaches.

In 2025, the WECHU implemented a predictive modeling program to enhance the existing beach monitoring and notification program. All municipalities and organizations responsible for managing public beaches in Windsor‑Essex County (WEC) were invited to participate. Four beaches (Colchester, Holiday Beach, Lakeshore Lakeview Park West, and Point Pelee Northwest) participated in the program and submitted daily turbidity and temperature readings through an online survey. The WECHU ran the model on weekdays, publishing results alongside regular weekly beach sampling and testing results on the website.

To assess the predictive modeling’s accuracy, effectiveness, and impact on public health decision-making, the WECHU conducted a comprehensive evaluation of the program.The evaluation of the program included stakeholder and beach user surveys, website analytics, and statistical validation of the predictive model. Stakeholders reported a high level of satisfaction with the training, resources, and data submission process provided by the health unit. Occasional challenges were noted, particularly related to staffing and environmental conditions at the beaches by participating stakeholders. The beach user survey results showed that beaches are valued as affordable, family-friendly spaces for recreation. There were varying levels of awareness of the beach monitoring program among the respondents. Most were aware of the existing Beach Water Testing website however none of the respondents were familiar with the predictive modeling component. Those respondents who were familiar with the program reported checking results on the Health Unit’s website or beach hotline before visiting the beach and adjusting plans accordingly, highlighting the value of timely reporting and the need to expand communication. Website analytics also confirmed substantial community engagement. More than 32,000 visits and over 313,000 interactions were recorded during the 2025 season. Residents actively engaged with interactive maps, weekly water test results, predictive forecasts, and educational content, demonstrating that the website serves as a decision-making support tool.

The predictive model demonstrated strong overall performance in estimating beach water quality risk, with statistical analysis showing a moderate correlation (0.46) between predicted and measured E. coli levels. When predictions were compared with the actual E. coli levels, 93% of all test points fell within the optimal or moderate validity categories. Optimal validity occurred when predicted risk aligned directly with required actions due to actual E. coli levels such as closures, warnings, or open status, while moderate validity reflected over or under‑estimations of the predicted risks. Only a small proportion of predictions (7%) fell into the poor validity category, and in those cases existing public health protocols ensured that it did not result in increased risk to beach users.

The WECHU sees this model as a useful and beneficial addition to the existing routine beach monitoring program in WEC. The next steps for the program include expanding participation among municipalities, strengthening communication strategies to increase public awareness of predictive forecasts, and refining the model to improve accuracy. These efforts ensure that the community receives accurate, up‑to‑date guidance that supports safe beach use across the region. With the goal of participation of all public beaches in the predictive modeling program, there is opportunity for shared commitment to improving the local beach water monitoring program.


INTRODUCTION 

The Windsor-Essex County Health Unit (WECHU) conducts beach water sampling and testing in accordance with Recreational Water Protocol, 2019 and following the Operational Approaches for Recreational Water Guideline, 2018. The program’s goal is to reduce the risk of waterborne illnesses and injuries by monitoring recreational water quality and restricting public beach use when unsafe conditions are present.

Seasonal beach monitoring includes surveillance of water and weather conditions, and potential pollutants, as well as weekly water sampling to test for bacteria called Escherichia coli (E. coli). These bacteria are found in the feces of humans, as well as all warm-blooded animals and birds. High levels of E. coli in beach water can be caused by animal waste, agricultural and stormwater runoffs as well as septic or sewage systems overflow. Routine beach water sampling begins at the end of May and ends before the Labour Day weekend in September. Water samples are taken weekly at eight local beaches (Sandpoint Beach, Lakeshore Lakeview Park West Beach, Point Pelee North West Beach, Seacliff Park Beach, Cedar Island Beach, Cedar Beach, Colchester Beach and Holiday Beach). In 2025, seven beaches were sampled due to closure of Sandpoint Beach. The samples are sent to the Public Health Ontario Laboratory in London for testing to determine the levels of E. coli bacteria present and the results are provided to the WECHU within 24-48 hours. 

The table below outlines the actions taken based on E. coli counts, measured in colony-forming units (CFU) per 100 mL, as per the Ministry of Health’s Recreational Water Protocol, 2019 (or current) and Operational Approaches for Recreational Water Guideline, 2018 (or current). 

E. coli (CFU/100 mL) Action Taken
≤200 Safe for swimming; no further action
201–999 Swimming not recommended; post Warning Sign
≥1000 Swimming not recommended; post Beach Closure Sign

The guideline identifies the E. coli levels of 200 CFU/100 mL or less as the acceptable concentration for routine recreation water use. As E. coli levels exceed this threshold it may increase the risk of illness among swimmers and results in a beach warning or closure. At 1000 CFU/100 mL or higher, water quality is considered unsafe, requiring immediate closure of the beach. These limits are designed to protect the public from associated health risks.

Swimming in contaminated water can result in a range of illnesses and infections with the most common being enteric illnesses that can cause diarrhea, vomiting, and other intestinal issues. In addition, being exposed to contaminated water can also lead to skin rashes, eye and ear infections, and respiratory infections. 

Routine beach monitoring has its limitations, most notably the delay between when the water sample is taken and when the laboratory results are received. Water quality results are generally posted online 24–48 hours after collection, during which the water conditions may change rapidly due to weather, bird activity, or the number of swimmers. Consequently, the posted results are not reflective of the current water quality conditions.

To overcome these challenges, the WECHU enhanced the beach water monitoring program with the introduction of a predictive model. The model uses a combination of historical data and real‑time environmental factors to generate estimates of water quality (i.e. E. coli counts) offering WEC residents timely and useful beach water results prior to visiting a beach.


WECHU’S PREDICTIVE MODELING PROGRAM

In 2025, the WECHU implemented a predictive modeling program to forecast daily E. coli levels at local public beaches. The model was built using WECHU’s historical water sampling data collected between 2014 and 2024, combined with environmental predictors such as wind speed, turbidity, water temperature, wave height, and rainfall. By analyzing these variables, the model generates daily estimates of the likelihood of elevated E. coli concentrations, reported as risk levels categorized as high, moderate, or low.

All municipalities and organizations that manage public beaches in WEC were invited to participate in the program, and an information session was held to explain its details. The program’s success relied on strong partnerships with these local stakeholders, as they were responsible for collecting daily turbidity levels and water temperature values. These readings were essential inputs for the predictive model. Five beaches agreed to participate:

  • Lakeshore Lakeview Park West Beach
  • Point Pelee North West Beach
  • Colchester Beach
  • Holiday Beach
  • Sandpoint Beach (Note: This beach was closed by the City of Windsor due to safety concerns and were unable to participate.)

To formalize participation, the WECHU signed Memorandums of Understanding (MOUs) with each municipality/organization which outlined the project’s objectives, methodology, expected outcomes, and the roles and responsibilities of all parties. The WECHU staff provided training and support materials including instruction sheets and a training video to ensure consistent and accurate data collection as well as the necessary equipment to conduct daily readings. Each day, staff at participating beaches submitted turbidity and water temperature readings through an online survey. These reading were used by the WECHU to run the predictive model on weekdays (excluding holidays) and the results were published on the Beach Water Monitoring webpage and updated on the Beach Hotline (ext. 1490). The results were displayed as:

Risk Level Action required
Low Risk Predicted results show water is safe for swimming
Moderate Risk Predicted results show water may not be safe for swimming and may pose a risk to your health
High Risk Predicted results show water is not safe for swimming and may pose a risk to your health.

Regardless of the model’s predictive results, if the beach was closed based on the weekly routine sampling, the predictive model’s output was marked as “Not Applicable” for that period, and the beach would remain closed. During closures, participating beaches continued to submit daily readings to support ongoing validation and refinement of the model. With the predictive model integrated into routine beach monitoring program, the next step was to evaluate its overall effectiveness, the accuracy of its predictions, and its impact on public health decision‑making.


Predictive Modeling Program Validation and Evaluation

The WECHU conducted a comprehensive evaluation of the predictive modeling program to determine how well the model forecasted daily E. coli levels and how useful it was in supporting public health decisions related to beach safety. The evaluation included three key components:

  • A Stakeholder Evaluation Survey - to gather insights on implementation and impact
  • A Beach User Evaluation Survey - to understand public perception and the usefulness of the forecasts
  • An Evaluation of the Validity of the Predictive Model determined how closely the predictions aligned with actual water sampling results

Stakeholder Evaluation Survey

The Stakeholder Evaluation Survey was conducted to collect feedback from the four municipal participant organizations of the predictive modeling program, focusing on the implementation and notification procedures. A short survey was emailed to key contacts to capture their experiences, challenges, and perceptions of the program’s effectiveness.

Three of the four stakeholders responded to the survey and the key findings are summarized below:

Training and Resources

  • All respondents indicated that the training sessions and reference materials provided by the WECHU including instructional slides, videos, and guidance documents were beneficial and informative.
  • The turbidity monitor was reported to be user-friendly, and the daily result submission process through the online survey was quick and easy.

Staffing for Water Sample Collection

  • Two respondents reported having between 2 and 4 designated staff members for daily turbidity testing and water temperature readings, while one respondent indicated having more than 5 staff members.

Challenges in Water Sample Collection

  • Two respondents experienced difficulties related to beach conditions (e.g. heavy rain) during water sample collection.
  • One respondent faced challenges due to an insufficient number of staff members assigned to the water collection process.

Result Submission 

  • Two out of three respondents consistently met the 12 pm submission deadline as outlined in the MOU. However, one respondent noted occasional missed submissions, primarily due to communication errors among staff working different shifts.

Perceived Benefits of the Program

  • Respondents acknowledged the predictive modeling program as a useful tool for predicting daily water quality results and providing same-day information to the public, emphasizing its value in public health communication.

Future Participation

  • One respondent confirmed its intention to continue participating in the predictive modeling program, showcasing a positive outlook toward this initiative, while two respondents expressed uncertainty regarding their future involvement, primarily due to staffing and resource limitations.

The evaluation findings highlight both the strengths of the predictive modeling program implementation and opportunities for further growth. The overall satisfaction among stakeholders suggests a successful rollout of this initiative, indicating a promising foundation for continued collaboration and future enhancements. While the program was well‑received, a few operational challenges were noted. Environmental conditions at certain sites occasionally impacted water sample collection, and staffing limitations affected the consistency of daily data submissions. 


Beach Users Evaluation Survey

The Beach Users Evaluation Survey was conducted to assess the public’s awareness of the WECHU’s beach water monitoring, including the predictive modeling program initiative. It also aimed to understand how this information influenced public decision-making regarding recreational water use.

The survey was administered in person by two WECHU staff members during July and August at two of the four participating public beaches: Pelee North West Beach and Lakeshore Lakeview Park West Beach. Completed responses were received from 18 respondents. The key findings are summarized below:

Beach use and motivations

Most respondents visited the beach with family citing recreation as the primary reason (33%), followed by staying cool in hot weather (26 percent) and that it was a low-cost activity (13%). Another 13% cited additional reasons, including proximity to home, having a day off, cleanliness, and low crowd levels. These findings demonstrate that beaches are valued not only as accessible spaces for leisure and family activities but also as practical, affordable options during hot weather.

Familiarity with the Beach Water Monitoring Program

  • The respondents were asked about their familiarity with the beach water monitoring program. Eight of the 18 respondents reported awareness of the routine beach water testing program, seven were not aware, and three did not provide an answer. None of the respondents indicated awareness of the predictive modeling component.
  • Among those aware of the regular beach water testing program (n=8), all reported checking water quality results on the Health Unit’s website or by calling the hotline before visiting the beach and changed their swimming plans based on the posted results. This highlights the value of timely water quality reporting and the need to strengthen communication so that more residents are aware of both regular beach water testing and daily predictive modeling results.

Interest in more information

  • The respondents were asked if they would like more information about the beach water testing program (n=16), and nine respondents expressed interest in receiving additional information about beach water quality. The WECHU staff provided details about the website and hotline, and distributed beach bags with hotline information.

Data collection during the 2025 season was limited to two beaches, primarily due to the WECHU’s staffing constraints and adverse weather conditions. The number of respondents who participated in the survey was low, which may be attributed to such factors as time of day or varying levels of public interest. As a result, the findings cannot be considered fully representative of all beach users across the region. As mentioned survey respondents’ awareness of the routine beach water testing program was mixed, and recognition of the new predictive modeling component was negligible. Strengthening communication about the predictive modeling program will be important to maximizing its public health impact. 

In addition to the beach user survey, website analytics of the Beach Water Testing page were conducted to assess community engagement and program reach. Results from the 2025 season (May 28 to August 27, 2025) demonstrated significant public interest, with over 32,000 visits generating more than 313,000 interactions. These interactions included viewing the interactive beach map, checking weekly test results, accessing predictive modeling forecasts, and reviewing educational content on water quality monitoring and beach safety. More than 33,000 direct clicks were made on specific features, confirming that residents were not only visiting the page but actively engaging with the information provided. The high level of engagement demonstrates the usefulness of this tool.    

Evaluation of the Validity of the Predictive Model

The validity of the predictive model was assessed by comparing the estimated E. coli levels and predicted risk categories (high, moderate, low) with the actual water sample results (beach closed, open with warning, or open with no warning). Routine sampling was conducted every Wednesday at all seven beaches as part of the established surveillance program, and additional samples were taken on Mondays at participating beaches to strengthen the dataset for model validation. All samples were analyzed for E. coli concentrations using standardized laboratory procedures. 

The model’s validity was rated as optimal, moderate, or poor based on how closely its predictions matched the actual water sample results.

Validity Categories Criteria (Actual E. coli Level vs. Predicted Risk)

Optimal validity

 

  • Beach closure (E. coli = 1000) when predicted risk = high
  • Warning (E. coli = 200-999) when predicted risk = moderate
  • Open/no warning (E. coli < 200) when predicted risk = low

Moderate validity

 

  • Beach closure (E. coli = 1000) when predicted risk is moderate
  • Warning (E. coli = 200-999) when predicted risk = high
  • Beach open (E. coli < 200) with no warning when predicted risk = moderate

Poor validity

 

  • Beach open (E. coli < 200) when predicted risk = high
  • Beach closed (E. coli = 1000) when predicted risk = low

During the 2025 monitoring season, a total of 55 beach assessments were conducted across 27 testing days. Routine weekly beach water sampling was done on Wednesdays accounted for 39 assessments, while supplementary beach water sampling was done on Mondays contributed to 16 assessments (except when holidays occurred). Each assessment included measurement of actual E. coli levels, turbidity, water temperature, accumulated rainfall over 48 hours, wind speed, and historical E. coli trends. These combined datasets provided a strong foundation for the validation of the predictive model. 

To evaluate how closely the model’s predictions matched the actual E. coli measurements, statistical analysis using Pearson correlation and path analysis was conducted. The correlation between predicted and observed E. coli levels was 0.46, which falls at the upper end of the moderate correlation range (0.30–0.49), indicating a strong relationship between model outputs and measured water quality. In addition to the correlation analysis, predicted risk categories were compared with actions warranted by actual E. coli levels. This comparison showed that 93% of all predictions demonstrated either moderate or optimal validity, reflecting strong overall alignment between predicted risk and observed conditions.

Validity Level Number of assessments  Result Description
Optimal validity 21/55   In 38% of assessments the model accurately predicted the same risk level that was later confirmed by water testing results.
Moderate validity 30/55   In 55% of assessments, the model estimated the risk as slightly higher or lower in relation to the actual water sample results. There were only 5 cases where the risk was underestimated as compared to actual water testing results.
Low validity 4/55 In 7% assessments, the model predicted the risk level did not match water testing results.

Note: Routine beach water sampling results took precedence over predictive modeling results and determined beach closures. 

Several limitations were identified in the implementation of the Predictive Modeling Program during the 2025 season. While participation from all eight beaches involved in routine beach water testing was anticipated, only four beaches participated in the program as noted previously. This reduced level of engagement constrained both the scope of testing and the validation of the model. Procedural challenges also emerged, particularly delays or failures by stakeholders in submitting turbidity and temperature readings by the agreed deadline of 12:00 p.m. Despite these limitations, the predictive model demonstrated strong validity and is able to provide reliable guidance for day-to-day beach management decisions. 


Next Steps

To ensure the continued success and expansion of the predictive modeling program, the WECHU will take the following actions:

Stakeholder Engagement

Strengthening engagement with municipalities to expand program participation is a priority. The goal is to have all local public beaches involved in the predictive modeling program. To achieve this goal, the WECHU will share the outcomes from the 2025 season with all local municipalities following the February Board of Health meeting, along with a formal invitation to participate in the program. Outreach will be directed toward municipalities and organizations choosing not to participate in the program through a one‑on‑one follow‑up meeting. These sessions will provide an opportunity to discuss concerns, the public health benefits associated with predictive modeling, and the broader value of water quality management for the WEC community.

Communication and Public Engagement

Strengthening communication and public engagement is important to maximize the program’s impact. Working in collaboration with the WECHU’s Communications Department, a comprehensive communication strategy will be developed to promote the predictive modeling program and support timely public decision-making regarding beach water safety. This will include exploring opportunities to update the existing website to improve user experience as well as developing targeted social media posts and radio advertisements about the predictive modeling program. Additionally, targeted outreach will also be directed to municipalities and organizations that run summer recreational programs, as well as to parents and guardians of students in schools located in high‑risk neighbourhoods recognizing that beach use is often a no‑cost recreational activity for many families in our community.

Model Refinement

To improve accuracy and reliability of the predictive model, the WECHU will review model performance and make necessary updates by enhancing the measurement of existing predictors. Efforts will also focus on addressing instances where predictions overestimate E. coli levels compared to actual measurements. 




List of links present in page
  1. https://www.wechu.org/board-health-meeting-agendas-and-minutes/february-2026-board-health-meeting-2025-predictive
  2. https://www.wechu.org/board-meetings/february-2026-board-health-meeting
  3. https://www.wechu.org/beaches/beach-water-testing
  4. https://www.health.gov.on.ca/en/pro/programs/publichealth/oph_standards/docs/protocols_guidelines/Recreational_Water%20Protocol_2019_en.pdf
  5. https://www.health.gov.on.ca/en/pro/programs/publichealth/oph_standards/docs/protocols_guidelines/Operational_Approaches_to_Rec_Water_Guideline_2018_en.pdf
  6. https://files.ontario.ca/moh-recreational-water-protocol-en-2019.pdf
  7. https://files.ontario.ca/moh-guidelines-operational-approaches-recreational-water-guideline-en-2018.pdf
  8. https://www.wechu.org/beaches/beach-water-testing