Listeria monocytogenes in Retail Delicatessens: An Interagency Risk Assessment—Risk Mitigations

Cross-contamination, improper holding temperatures, and insufficient sanitary practices are known retail practices that may lead to product contamination and growth of Listeria monocytogenes. However, the relative importance of control options to mitigate the risk of invasive listeriosis from ready-to-eat (RTE) products sliced or prepared at retail is not well understood. This study illustrates the utility of a quantitative risk assessment model described in a first article of this series (Pouillot, R., D. Gallagher, J. Tang, K. Hoelzer, J. Kause, and S. B. Dennis, J. Food Prot. 78:134–145, 2015) to evaluate the public health impact associated with changes in retail deli practices and interventions. Twenty-two mitigation scenarios were modeled and evaluated under six different baseline conditions. These scenarios were related to sanitation, worker behavior, use of growth inhibitors, cross-contamination, storage temperature control, and reduction of the level of L. monocytogenes on incoming RTE food products. The mean risk per serving of RTE products obtained under these scenarios was then compared with the risk estimated in the baseline condition. Some risk mitigations had a consistent impact on the predicted listeriosis risk in all baseline conditions (e.g. presence or absence of growth inhibitor), whereas others were greatly dependent on the initial baseline conditions or practices in the deli (e.g. preslicing of products). Overall, the control of the bacterial growth and the control of contamination at its source were major factors of listeriosis risk in these settings. Although control of cross-contamination and continued sanitation were also important, the decrease in the predicted risk was not amenable to a simple solution. Findings from these predictive scenario analyses are intended to encourage improvements to retail food safety practices and mitigation strategies to control L. monocytogenes in RTE foods more effectively and to demonstrate the utility of quantitative risk assessment models to inform risk management decisions.

Sampling surveys of ready-to-eat (RTE) foods suggest that products sliced or prepared at retail delicatessens are more frequently and more heavily contaminated with Listeria monocytogenes and present a substantially higher risk than products sliced and packaged at the manufacturer level (1,4,7,11,21). L. monocytogenes strains are found to be widely distributed and to persist in retail facilities (13,23,26). Common retail practices may lead to product contamination from contaminated environmental sites, cross-contamination from one RTE product to another, or both (10,16,22). Additional risk factors include the impact of improper holding temperatures on the growth of L. monocytogenes and insufficient sanitary practices that may contribute to L. monocytogenes contamination of RTE foods at retail (32,33). The relative importance of these different factors on the potential risk of invasive listeriosis from the consumption of products sliced or prepared at retail is nevertheless not well understood. Therefore, a quantitative risk assessment (QRA) was conducted collaboratively by the U.S. Department of Health and Human Services, U.S. Food and Drug Administration (FDA), Center for Food Safety and Applied Nutrition and the U.S. Department of Agriculture (USDA), Food Safety and Inspection Service (FSIS), in consultation with U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. A primary objective of this QRA was to assess the risk of foodborne listeriosis associated with current practices in retail delis. However, the interest of this QRA relied less in the prediction of the risk of invasive listeriosis than on the extent to which this risk may increase or decrease significantly when some changes in the retail deli practices are applied. Indeed, the evaluation of putative risk management actions is often based on comparison of the baseline risk estimate with a forecast risk that could result from pursuing various alternative strategies (sometimes called ''what-if'' scenarios) (9). In this second article of the series, we conduct a scenario analysis using the model described previously (18) to examine how the risk may be mitigated by changes in practices that limit L. monocytogenes growth or prevent cross-contamination of RTE foods prepared in retail delis.

MATERIALS AND METHODS
Description of model. The discrete-event model developed for this study (i.e. the ''virtual deli'' model) has been described extensively in the first article of this series (18). In brief, the model mimics an operating retail deli department and tracks the L. monocytogenes that may potentially be present on various environmental sites (e.g. slicer, utensils) and on selected food products (i.e. deli meat, deli cheese, or deli salad). The model simulates worker behavior, the contamination and cross-contamination of RTE foods, and the growth or decline of L. monocytogenes in the deli-prepared RTE foods, and then it predicts the subsequent risk of invasive listeriosis among consumers.
Description of baseline retail conditions. Two sources of L. monocytogenes contamination were considered in the model: (i) contamination on incoming RTE foods entering the deli (e.g. chubs of meat, blocks of cheese, or bulk salad) and (ii) contamination on retail environmental surfaces or niches. The approach used was to evaluate and compare the public health impact of various food safety mitigations under six different baselines that may characterize different operational conditions in retail delis. In brief, these six baseline conditions were as follows: (i) a retail deli with multiple niches or regular environmental contamination of nonfood contact surfaces (NFCS) (baseline denoted as ''multiple niche 100W''); (ii) a retail deli with no niches or environmental L. monocytogenes transfer (''no niche''); (iii) a retail deli with no niche or environmental L. monocytogenes transfer, with one incoming RTE product that supports growth and is contaminated at L. monocytogenes levels higher than those observed in USDA-FSIS monitoring (30) (''highly contaminated incoming chubgrowth''); (iv) a scenario similar to the scenario iii where the product does not support growth (''highly contaminated incoming chub -no growth''); (v) a retail deli with multiple niches, compliant with temperature control (''multiple niche and temperature control''); and (vi) a retail deli without niches, compliant with temperature control (''no niche and temperature control''). A more complete description of these retail baseline conditions is provided in Table 1 and in the companion article (18). Within each of the six retail baseline conditions, we used the model to evaluate the public health impact of a variety of what-if scenarios related to changes in sanitary practices, worker behaviors, product formulations, crosscontamination, and impact of product storage temperature and duration.
Description of tested food safety mitigation options. Fourteen specific risk management questions were developed ( Table 2). Some of the risk management questions were generated by the Interagency Retail L. monocytogenes Risk Assessment Workgroup, whereas other questions were the outcome of discussions with stakeholders. These risk management questions were then translated into a list of 22 what-if scenarios to be evaluated within the risk assessment model. Some of the what-if scenarios directly addressed a specific risk management question. For example, the scenario ''Retail deli workers clean deli NFCS as if they were food contact surfaces (FCS), i.e., every 4 h in accordance with the 2013 FDA Food Code'' directly addresses the question ''What if scale touch pads, refrigerator and deli case handles, and other frequently touched NFCS were considered FCS and were therefore cleaned and sanitized at a minimum frequency as per FDA Food Code (34) requirements?'' In other instances, additional scenario analysis was conducted to evaluate the public health impact of specific risk management questions more effectively. For example, the what-if scenario ''Retail deli workers change gloves for every sale of RTE products'' was evaluated to address the question ''What is the potential public health impact of increasing the use of single service gloves in the retail delis?'' An additional scenario, ''Retail deli workers do not use gloves when serving customers,'' also was evaluated to better characterize the impact of this worker behavior practice. Finally, some what-if scenarios were evaluated to characterize and describe the model behavior. For example, a scenario without any cross-contamination in the deli was tested to explore the impact of cross-contamination in this setting, even though no risk management option could achieve this goal consistently. Similarly, a scenario without any sanitation was tested to evaluate the overall impact of sanitation, No niche A retail deli without any niches with an incoming RTE product that supports the growth of L. monocytogenes and has a mean incoming L. monocytogenes concentration increased from the observed À9.2 to À5 log CFU/g. Highly contaminated incoming chub -growth A retail deli without any niches with an incoming RTE product that does not support the growth of L. monocytogenes and has a mean incoming L. monocytogenes concentration increased from the observed À9.2 to À5 log CFU/g. Highly contaminated incoming chub -no growth A retail deli with multiple niche 100W (see above) that maintains the temp of the deli case to 58C ( 418F). Multiple niche and temp control A retail deli without any niches that maintains its deli case to 58C ( 418F).
No niche and temp control What would be the potential impact on the prevalence of L. monocytogenes in RTE products sold in retail delis and on the corresponding mean risk of invasive listeriosis of practicing more frequent or more extensive cleaning procedures for FCS and/or NFCS than currently specified in the 2013 FDA Food Code (34)?
Increase the effectiveness of retail deli cleaning from simply washing to washing and sanitizing (i.e. from an average log reduction from a Pert(1.5, 0.5, 0) to a Pert(8, 6, 1.5), see (18)).

Wash and sanitize
Doubling the number of retail deli sites sporadically cleaned from four to eight.

NFCS as FCS
What is the potential public health impact of increasing the use of single-service gloves in the retail delis?
Retail deli workers do not use gloves when serving customers.

No glove
Retail deli workers change gloves for every sale of RTE products.

Gloves every serving
What would be the potential public health impact of ''preslicing'' all RTE products vs ''slicing to order'' (hypothesis: less crosscontamination occurring in morning before other cross-contamination events).
Retail deli workers preslice all chubs of RTE product (deli meat and deli cheese) in the morning after cleaning. (A quantity equal to the median of the daily sales is presliced every morning. When a consumer orders a RTE product, the food worker serves the presliced RTE product, until the presliced quantity is all sold. If needed, additional RTE product is sliced to order. At the end of the day, the remaining presliced RTE product is discarded.)

Preslice
What would be the potential public health impact of using separate slicers and/or separate counters for RTE products that permit growth of L. monocytogenes and for RTE products that do not?
Retail deli workers use a separate slicer for RTE products that support growth of L. monocytogenes vs those RTE products that do not.

Separate slicer
Retail deli workers use of a separate slicer and a separate deli case for RTE products that support the growth of L. monocytogenes vs those RTE products that do not. Retail deli workers collect the slices of RTE products directly on tissue paper rather than on his or her gloves.

Do not slice on gloves
What would be the impact if all (or no) RTE products (e.g. deli meats, deli salads) coming into the deli were formulated with growth inhibitors?
Reformulate all RTE products sold at the retail deli that would otherwise support L. monocytogenes growth to include growth inhibitors to restrict the growth (same GI a formulation as cured ham with GI).
All GI Reformulate all RTE products that support L. monocytogenes growth that are sold at the retail deli to not include GI to restrict L. monocytogenes growth.
No GI which does not correspond to any risk management option. Tables 3 through 8 provide a description of each of the tested scenarios as well as how they were translated in the model.

Evaluation of food safety mitigation option impact.
Each what-if scenario was evaluated for each of the six retail baseline conditions. Some additional scenarios were explored in more detail to understand the impact of an increased L. monocytogenes contamination in incoming products and the impact of home storage on the predicted risk.
Each what-if scenario was simulated by changing one or a few parameters from the baseline model. As an example, the ''noglove'' scenario could be evaluated from the baseline condition by setting the probability for a food worker to use gloves to 0. A brief description of the changes made from the baseline models is provided (in parentheses) in Tables 3 through 8. For each retail baseline condition, the mean risk of listeriosis per serving was evaluated in 30 simulations, with each simulation modeling 100 stores that each provided 1,000,000 servings (total of 10 8 servings per simulation). Each of the 30 simulations was performed using a different random seed. Each what-if scenario was evaluated with one simulation of 100 stores, with each providing 1,000,000 servings for every baseline condition. The percentage of change in the mean risk per serving for the susceptible population in a given scenario relative to the median from the 30 estimations of this mean risk in the corresponding baseline condition was evaluated and used as an estimate of the impact of the given scenario in the current baseline. If the mean risk per serving for the susceptible population in the what-if scenario was lower than the 2.5th percentile or greater than the 97.5th percentile of the 30 estimations of this mean risk for the baseline, the what-if scenario was deemed significantly different from the baseline. This procedure allowed us to identify those changes that are significant and not due to random uncertainty in the model.

RESULTS AND DISCUSSION
The risk assessment model was specifically designed as a virtual deli to evaluate the impact of changes in retail practices and potential mitigation options. As has been discussed previously (9), such evaluations are best studied through what-if scenarios that simulate the impact of specific changes in handling practices and interventions on the predicted change in public health risk and that allow for comparisons between baseline risk estimates and the forecasted risk that could result from the implementation of various alternative mitigation strategies. Notably, some of the evaluated scenarios are hypothetical in the sense that they do not represent actual risk management options, but are instead pursued to understand the risk assessment model What if practices were in place so that no crosscontamination occurred in delis (i.e. no additional L. monocytogenes added to incoming RTE products)?
Scenario where L. monocytogenes crosscontamination in the retail deli would only result from the deli slicer (i.e. set crosscontamination transfer coefficients to 0 for all sites except the slicer).

Transfers to 0
Scenario where there is no L. monocytogenes cross-contamination in the retail deli (i.e. set cross-contamination transfer coefficients to 0; i.e., no cross-contamination occurs for all sites, including the slicer).
Transfers and slicer to 0 What would be the potential public health impact of lowering the level of environmental contamination of FCS?
Reduce transfer of L. monocytogenes among RTE products, FCS, and NFC (i.e. reduced transfer coefficients by 50%) in the retail deli.

Lower env cont
What is the potential public health impact of bacterial growth in retail delis?
Set the retail deli case temp for all retail delis to 58C (418F) (i.e. in compliance with the 2013 FDA Food Code) rather than using real-world deli case temps reported by EcoSure. What would be the impact of shortening the time a RTE product can be used in a deli department?
Retail delis reduce the length of time RTE products are held before they are sold or disposed from 7 to 4 days.

Shorten time in retail deli
What would be the potential public health impact if the level of L. monocytogenes contamination is reduced in RTE foods coming into the retail deli?
Lower the mean incoming L. monocytogenes concentration on all RTE products from the observed mean of À9.2 log CFU/g to a mean of À9.5 log CFU/g.

Reduce level
a GI, growth inhibitor. and the intricate impacts of selected changes in the deli system. These scenarios should be considered as sensitivity analyses rather than evaluations of the actual risk management option (9). In Table 3, each column represents one of the six different baseline conditions that were tested, and each row shows the impact of a selected mitigation option. Tables 4 through 8 are organized similarly. The impact of each scenario was evaluated for the susceptible population only. Although the predicted absolute risks differed between the susceptible populations and the general population, the predicted relative change in risk was similar for both populations. This observation is linked to the linearity of the Food and Agriculture Organization of the United Nations, World Health Organization dose-response model for L. monocytogenes (8). We report here only the results for the susceptible population. The impact of selected mitigations (in each row) is shown as the percentage of change in the risk per serving relative to the respective baseline estimate (hereafter relative risk). The scenarios are organized by topic in the tables as follows: sanitation (Table 3), worker behavior (Table 4), use of growth inhibitors (Table 5), cross-contamination (Table 6), storage temperature control (Table 7), and reduction in the level of L. monocytogenes on incoming RTE products (Table 8). In these tables, positive values represent an increase in the predicted absolute risk for the susceptible population, compared with the corresponding baseline. Negative values represent a decrease compared with the corresponding baseline in the predicted relative risk per serving. The relative effectiveness of the scenarios applied to a specific baseline condition can be assessed by reading down the columns of Tables 3 through 8.
The relative impact on the predicted risk of listeriosis from a change in retail practice across different operating conditions in different types of retail delis can be assessed by comparing estimates across different cells in each row in Tables 3 through 8. For correct interpretation of the tables, it is important to consider the order of magnitude of the mean risk per serving for the susceptible population predicted for the applicable baseline conditions, when no specific scenario is considered (i.e. 1.7 3 10 À7 for the multiple niche 100W baseline, 1.4 3 10 À7 for the no niche baseline, 16.6 3 10 À7 for the highly contaminated incoming chub -growth baseline, 2.8 3 10 À7 for the highly contaminated incoming chub -no growth baseline, 1.5 3 10 À7 for the multiple niche and temperature control baseline, and 1.2 3 10 À7 for the no niche and temperature control baseline). Importantly, analysis of the baseline  highly contaminated incoming chub -growth, 16.6 3 10 À7 ; highly contaminated incoming chub -no growth, 2.8 3 10 À7 ; multiple niche and temp control, 1.5 3 10 À7 ; no niche and temp control, 1.2 3 10 À7 . b Results characterized as significant, because they are outside the 95% confidence interval of the corresponding retail baseline condition. results indicates that the risk of listeriosis was strongly correlated with the contamination of RTE products that support L. monocytogenes growth, regardless of the origin of the contamination (e.g. contaminated as an incoming product, contaminated through cross-contamination from the retail environment, contaminated via other RTE products) or whether these food vehicles support L. monocytogenes growth (18). This finding is relevant in the interpretation of mitigation options.
Sanitation-related scenarios. The results shown in Table 3 indicate that retail sanitation practices are key drivers in reducing the predicted risk of listeriosis from consumption of RTE foods prepared or handled in delis.
When sanitation activities were modeled as absent (''no sanitation''), the predicted risk increased by 50.2% (i.e. multiple niche and temperature control column). A study of the pathways by which L. monocytogenes was introduced into the deli shows that the elimination of bacteria through sanitation practices was a significant factor. Indeed, the smallest predicted increase in risk from omitting sanitation was 2.9% (highly contaminated incoming chub -growth column). In this scenario, the impact of sanitation was outweighed by the additional bacteria stemming from highly contaminated incoming product and growth of L. monocytogenes in the product while in the retail deli. However, on the basis of the results provided here, it seems that once a reasonable level of sanitation has been reached, a Mean risk per serving for each baseline conditions for the susceptible population. Multiple niche 100W, 1.7 3 10 À7 ; no niche, 1.4 3 10 À7 ; highly contaminated incoming chub -growth, 16.6 3 10 À7 ; highly contaminated incoming chub -no growth, 2.8 3 10 À7 ; multiple niche and temp control, 1.5 3 10 À7 ; no niche and temp control, 1.2 3 10 À7 . b Results characterized as significant, because they are outside the 95% confidence interval of the corresponding retail baseline condition. factors such as modifying any of the single sanitationrelated practices individually, such as cleaning more effectively (i.e. ''wash and sanitize'' scenario, significant in only one baseline condition), increasing the number of sporadic cleaning sites (i.e. ''clean 8 sporadic'' scenario, significant in only one baseline condition), or not conducting any sporadic cleaning (''no sporadic cleaning'' scenario, not significant), has limited impact by itself on the predicted relative risk in each retail deli condition studied (Table 3).
It was observed that NFCS, such as scale touch pads and refrigerator and deli case handles, usually are not cleaned and sanitized as frequently as FCS (15). Treating NFCS as FCS for cleaning and sanitizing purposes had little impact on the predicted risk (i.e. ''NFCS as FCS'' scenario). The only retail condition where the risk reduction was statistically significant highly contaminated incoming chub -growth, 16.6 3 10 À7 ; highly contaminated incoming chub -no growth, 2.8 3 10 À7 ; multiple niche and temp control, 1.5 3 10 À7 ; no niche and temp control, 1.2 3 10 À7 . b GI, growth inhibitor. c Results characterized as significant, because they are outside the 95% confidence interval of the corresponding retail baseline condition. a Mean risk per serving for each baseline conditions for the susceptible population. Multiple niche 100W, 1.7 3 10 À7 ; no niche, 1.4 3 10 À7 ; highly contaminated incoming chub -growth, 16.6 3 10 À7 ; highly contaminated incoming chub -no growth, 2.8 3 10 À7 ; multiple niche and temp control, 1.5 3 10 À7 ; no niche and temp control, 1.2 3 10 À7 . b Results characterized as significant, because they are outside the 95% confidence interval of the corresponding retail baseline condition. for this scenario was for a retail deli without any niches that implemented temperature control. In this situation where L. monocytogenes levels are low and growth is limited, the additional cleaning was predicted to be beneficial in controlling this already low baseline risk of 1.2 3 10 À7 .
Worker behavior-related scenarios. The impact of simulated changes to worker behavior on the predicted risk varied, depending on the baseline retail deli condition, as shown in Table 4. For example, using separate slicers in a retail deli that had multiple niches (i.e. multiple niche 100W column and ''separate slicer'' scenario) reduced the predicted relative risk by 6.3%. If workers did not use gloves that could be routinely discarded (i.e. no-glove scenario), the predicted relative risk increased from 5.1 to 8.5%. In other cases, the public health benefits of some ; highly contaminated incoming chub -growth, 16.6 3 10 À7 ; highly contaminated incoming chub -no growth, 2.8 3 10 À7 ; multiple niche and temp control, 1.5 3 10 À7 ; no niche and temp control, 1.2 3 10 À7 . b Results characterized as significant, because they are outside the 95% confidence interval of the corresponding retail baseline condition. c NA, not applicable. monocytogenes concentration on all RTE products from the observed mean of À9.2 log CFU/g to a mean of À9.5 log CFU/g À21.
a Mean risk per serving for each baseline conditions for the susceptible population. Multiple niche 100W, 1.7 3 10 À7 ; no niche, 1.4 3 10 À7 ; highly contaminated incoming chub -growth, 16.6 3 10 À7 ; highly contaminated incoming chub -no growth, 2.8 3 10 À7 ; multiple niche and temp control, 1.5 3 10 À7 ; no niche and temp control, 1.2 3 10 À7 . b Results characterized as significant, because they are outside the 95% confidence interval of the corresponding retail baseline condition. interventions seem to have been cancelled out by other factors.
The lack of glove use (i.e. no-glove scenario) consistently increased the predicted risk across all retail deli types, often at statistically significant levels (þ1.2 to þ7.0%). Therefore, retail glove use probably contributes to L. monocytogenes control and is even more critical for other foodborne pathogens, such as norovirus or Shigella that are associated with transmission by food workers via the fecaloral route (27). Changing gloves after every serving (i.e. ''gloves every serving'' scenario) did not result in a significant change in the predicted risk, but gloves are currently already changed frequently, for~65% of servings in baselines (15,18).
Preslicing generally increased the predicted risk of listeriosis, often substantially, and especially if a contaminated growth-supporting RTE product was being sliced (i.e. ''preslice'' scenario). This trend was due to contamination of the slicer with a contaminated chub during the preslicing process. A high number of RTE product servings are then cross-contaminated, leading to a higher predicted risk. A retail deli with an incoming contaminated chub that does not support the growth of L. monocytogenes was the only situation in which preslicing led to a significantly lower predicted risk compared with the baseline. In this situation, preslicing leads to a distribution of the bacteria to the same category of (nongrowth) RTE products, rather than to various (growth and nongrowth) RTE product that is sliced throughout the day and limits potential cross-contamination of the RTE product that supports the growth, a key driver of the predicted risk.
The risk assessment model generally predicted that having separate slicers for products that support L. monocytogenes growth and those that do not had minimal impact in mitigating the risk of listeriosis (i.e. separate slicer scenario). The only exception was observed in the baseline where one incoming product that does not support growth of L. monocytogenes was contaminated at levels higher than those observed in USDA-FSIS monitoring (30). Additional studies of the outputs of the model suggest that, in the tested situation including two slicers in the deli, the slicer for incoming nongrowth product was eventually used for more products than in the baseline scenarios where slicers were not separated; as a consequence, more RTE product was incidentally cross-contaminated with L. monocytogenes. The resulting predicted risk was therefore higher than when slicers were not separated. However, this counterintuitive result would not necessarily be generalizable to larger stores with a large number of slicers. Importantly, such model behavior points out the complexity of the interaction among the various elements in the model and emphasizes the need for in-depth analysis of model outputs and extensive sensitivity analysis.
Preventing contact between the case handle and hands or gloves resulted in no significant change in the predicted risk in the baselines studied (i.e. ''no-contact glove case'' scenario). Similarly, slicing RTE product onto deli paper rather than onto gloves (i.e. ''do not slice on gloves'' scenario) had little impact on the relative risk predicted by the model, with the exception of the multiple niche and temperature control baseline condition, where the risk was significantly increased. Slicing onto gloves was a behavior observed during the observational study (15). Usually, no reduction in the predicted risk was associated with eliminating this practice, although risk was reduced with some of the other mitigation options that are directed at preventing cross-contamination. Together, these results suggest that, within a given baseline scenario, no single cross-contamination event had a major impact on the predicted risk with the exception of the cross-contamination within the slicer (see ''Cross-contamination-related scenarios''). These results suggest that a combination of multiple cross-contamination events results in an increased risk from RTE food prepared at retail. It may be difficult, but necessary, to prevent L. monocytogenes cross-contamination through a combination of sanitary controls (see ''Crosscontamination-related scenarios'').
Growth inhibitor-related scenarios. Of all the mitigation options evaluated using this QRA, the formulation of RTE products with growth inhibitors had the greatest impact on the predicted relative risk ( Table 5). The use of growth inhibitors in all products almost eliminated the predicted risk of listeriosis, reflected by reductions in relative risk that ranged from À94.4 to À97.5%. Although it is probably not feasible to formulate all RTE products with growth inhibitors because of adverse impacts on the quality of the products (e.g. taste), this level of predicted relative risk reduction (i.e.~95%) is a significant finding and emphasized that this mitigation option may nearly eliminate the risk. Growth inhibitors continue to prevent growth even after the RTE product leaves the retail deli and therefore directly limit bacterial growth during home storage, a key driver of the listeriosis risk (6,8,19,28). The baseline scenarios consider that products in the retail deli are a mixture of products that include growth inhibitors and products that do not. In a comparison scenario in which no products in the retail deli contained growth inhibitors (''no GI''), the predicted risk nearly doubled to 184.1 and 191.5% compared with the baselines. The only apparent exception is in the highly contaminated incoming chub -growth baseline, where the estimated relative increase in predicted risk was only 35.1%. This relatively low value is somewhat misleading, because the predicted absolute risk for this baseline was already almost 10 times higher than for other baseline results. These results confirm the overwhelming importance of limiting the contamination of RTE products that support the growth of L. monocytogenes to mitigate the risk of listeriosis, as has been consistently observed in other studies (8,28,29).
Cross-contamination-related scenarios. Table 6 results show that controlling cross-contamination in retail delis has a significant impact on the predicted risk of listeriosis. In the what-if scenarios in which cross-contamination did not occur in the retail deli (i.e. ''transfers and slicer to 0'' row), the predicted relative risk reduction was significant for all baseline conditions (i.e. from À9.5 to À60.8%). However, when L. monocytogenes transfers from all sites were eliminated, but transfers from the slicer were not eliminated (transfers to 0 scenario), the predicted relative risk was not reduced. This result further highlights the importance of the slicer in retail delis as a primary source of cross-contamination at retail. The importance of the slicer as a point of cross-contamination has been demonstrated experimentally (3,12,16,17,24,25,35). Our results predict a significant risk reduction if all cross-contamination events are eliminated in a deli department setting-up to 60% decrease in the relative risk. This suggests that management practices that prevent cross-contamination in retail delis would substantially mitigate the risk of listeriosis from retail-prepared RTE foods.
Control of storage temperature control and duration. Improving temperature controls in display cases to limit growth resulted in a lower predicted relative risk across all tested retail deli conditions ( Table 7). Note that the ''temperature ¼ 58C'' (418F) alternative represents an increase in the temperature for some of the RTE products (i.e. those stored at a temperature of ,58C) compared with the baseline. By setting the temperature of display cases to 58C, the overall predicted risk was still reduced. The hypothetical ''no growth'' scenario results in up to a 20% reduction in predicted risk.
For the scenario in which the RTE foods were held at the recommended temperature (i.e. ''temperature 58C''), the predicted reduction in risk was roughly the same as the reduction associated with holding RTE foods at temperatures that completely prevents growth of L. monocytogenes (i.e. no growth). The model studied the effect of full compliance with the 2013 FDA Food Code (34) temperature requirements on the predicted risk, compared with typical temperature conditions observed in a study by EcoSure (5). The model predicts that maintaining products in the deli display at a temperature recommended by the FDA Food Code (34) can prevent almost all of the additional risk linked to bacterial growth at retail. The effect was greatest in a retail deli without any niches. The model predictions suggest that a reduction of the risk of listeriosis could be achieved by better compliance with recommended RTE product storage temperatures in the deli environment. The FDA's 2008 Retail Risk Factor Study revealed that the failure in delis to control RTE product holding temperatures and times was the risk factor with the highest ''out of compliance'' percentage (32). For example, in 60% of the 98 retail delis studied by the FDA in 2008, at least one observation was made in which food requiring temperature control was not held at 58C or below, as specified in the FDA Food Code (32). This finding further demonstrates the importance of controlling the growth of L. monocytogenes in RTE products, as suggested in previous risk assessments (8,28,29).
Shortening the maximum time a refrigerated RTE product that supports growth of L. monocytogenes is allowed to remain on hand in the retail deli after opening or preparation from 7 to 4 days was predicted to have little effect on the risk. Under the current model, the time from when the chub is opened until it is completely sold is generally shorter than 7 days, as required by regulations modeled after the FDA Food Code. Importantly, the model does not currently simulate refrigerated storage of product during transport or in reach-in and walk-in refrigerators before the chub or deli salad bulk is opened.
Concentrations of L. monocytogenes in incoming product. Reducing the concentration of L. monocytogenes on incoming RTE products significantly reduced the resulting risk of listeriosis from retail prepared foods (i.e., ''reduce level'' scenario, Table 8). A decrease in L. monocytogenes concentration on incoming RTE products by a factor of 2 (from a lognormal distribution with a mean of À9.5 log CFU/g to a lognormal distribution with a mean of À9.2 log CFU/g, with the standard deviation being maintained at 2.9 log CFU/g) was predicted to reduce the risk of listeriosis from retail prepared RTE foods by À10 to À24%. In this scenario, incoming RTE products contaminated with L. monocytogenes represent one of the major sources of contamination of retail FCS and subsequent cross-contamination of other RTE foods. Even in retail deli settings in which L. monocytogenes is already present in niches, a reduction in the levels of L. monocytogenes on incoming RTE products was predicted to have a substantive impact in reducing the risk from deli-prepared RTE foods. The model predicts that controlling even low levels of L. monocytogenes on incoming RTE products is important to mitigating the risk of listeriosis from deli-prepared RTE foods. What-if scenarios were also conducted to explore the public health impact of higher levels of L. monocytogenes on incoming RTE products. In response to public comments on the QRA, additional scenarios were conducted to evaluate the impact of high contamination rates (incoming RTE products that support growth or that do not support growth with 100 CFU/g of L. monocytogenes) on the predicted risk (14,31). The concentration of L. monocytogenes on incoming RTE products was set from a lognormal distribution with mean À9.2 log CFU/g of L. monocytogenes (standard deviation of 2.9 log CFU/g of L. monocytogenes [baseline]) to exactly 100 CFU/g of L. monocytogenes (Table 9). Increased contamination of RTE products that support growth increases the average predicted risk from 1.4 3 10 À7 to 300 3 10 À7 per serving. This increase in the risk is primarily the result of high levels of L. monocytogenes on incoming RTE product, because the predicted risk is unchanged if there is no retail cross-contamination (300 3 10 À7 , Table 9). In contrast, when the incoming RTE products do not support L. monocytogenes growth, retail cross-contamination contributes to a substantial increase in the risk of listeriosis, from 1.2 3 10 À7 to 66 3 10 À7 . Without retail cross-contamination, increasing the level of L. monocytogenes in RTE products that do not support L. monocytogenes growth does not contribute much to the predicted risk of listeriosis (1.2 3 10 À7 ) compared with the baseline (1.1 3 10 À7 ). Note that these additional scenarios assumed that the proportion of incoming RTE products with !100 CFU/g would shift from 0.01 to 100%. This scenario might be considered somewhat extreme in that it is unlikely that all incoming product would be contaminated at 100 CFU/g. Overall, these scenarios demonstrate that controlling the level of L. monocytogenes on incoming RTE products, both those that support L. monocytogenes growth and those that do not, is critical to mitigating the risk of listeriosis. The results also show that even RTE products that do not support L. monocytogenes growth could substantially contribute to the risk of listeriosis as a result of cross-contamination in the retail deli.
Consumer storage practices. An additional evaluation of the impact of consumer storage practices on the predicted risk was conducted. In a first scenario, products were kept at home during a time similar to that in the baseline, but all home refrigerators were consistently held at a temperature below 58C. In a second scenario, the home refrigerator temperatures were set back to the temperature used in the baseline (5,20), but all meat products and salads were consumed within 3 days after purchase and all cheese products were consumed within 4 days after purchase. All other parameters were set as in the baseline conditions. These scenarios were run for two baseline retail conditions: multiple niche 100W and no niche. The predicted risk of listeriosis was almost eliminated if the products were stored at home less than 3 to 4 days or if home-refrigerator temperatures were lower than 58C (Table 10). Importantly, these scenarios assumed a complete implementation of the recommendations by all consumers (i.e. home-refrigerator temperature in the first scenario and use-by days in the second scenario), which is unlikely to be met. The reductions in the risk were therefore likely to be overestimated, compared with what would be expected based on imperfect implementation of the recommendations in practice. Nonetheless, these scenarios confirm the dramatic impact of home-storage duration and temperature on the risk of listeriosis. Similar findings have been reported in other national and international risk assessments on listeriosis (8,29), namely, that the risk of listeriosis was driven by storage for a longer period and at higher-than-recommended temperatures for contaminated RTE products that supported growth. In addition, this present risk assessment shows that the risk can be reduced by avoiding contamination and cross-contamination of these products in advance.
In conclusion, some of the essential duties of risk managers are to determine which hazards or practices present more than an acceptable level of risk to society and to consider what control options may be available (2). The ultimately chosen mitigation options need to be effective and efficient in mitigating risk and must be feasible to implement. In a retail setting, numerous parameters impact the potential risk of contamination and growth of L. monocytogenes in RTE products. Experimental studies in delis or mock delis provide useful specific information for QRA model calibration and validation. When integrated into QRAs, such as this virtual deli model, we can evaluate the public health impact of current practices and predict the public health impact of changes in practices. The development of this QRA, mimicking existing delis operations and practices, represents a complement to experimental studies and allows for the testing and comparison of the impact of various and interrelated risk factors.
This QRA represents the first large-scale effort to model L. monocytogenes cross-contamination at retail in its entirety. The use of the QRA, through the conduct of what-if scenario analyses, provides the first in-depth insight into the extent to which changes in practices and mitigation options impact the risk of listeriosis from deli-prepared RTE TABLE 9. Predicted absolute risk of invasive listeriosis for the susceptible population per serving of RTE food sliced or prepared and sold at retail delis according to various scenarios under the no niche baseline Scenario Baseline (with cross-contamination) Transfers and slicer to 0 (without cross-contamination) Predicted risk per serving, no niche baseline 1.4 3 10 À7 1.1 3 10 À7 All incoming products have a concentration of 100 CFU/g 300 3 10 À7 300 3 10 À7 Only incoming products that support growth have a concentration of 100 CFU/g 300 3 10 À7 300 3 10 À7 Only incoming products that do not support growth have a concentration of 100 CFU/g 66 3 10 À7 1.2 3 10 À7 Predicted risk per serving, baseline conditions (from (18)) 1.7 3 10 À7 1.4 3 10 À7 Products are kept in home as in the baseline, but all home refrigerators have a temperature lower than 58C (418F) 0.0042 3 10 À7 0.0027 3 10 À7 Home-refrigerator temperature distribution as in the baseline, but all meat products and salads are used within 3 days after purchase, and all cheese products are used within 4 days after purchase 0.021 3 10 À7 0.014 3 10 À7 foods. The risk assessment model contributes to our understanding and to the comparison of the relative impact of L. monocytogenes transmission, survival, and growth in the retail environment and was used to evaluate how retail practices may impact the predicted risk of listeriosis. The approach used evaluated the public health effect of various changes in practices under six different baseline conditions that may characterize a retail deli and the RTE food it serves. Although some risk mitigations had a consistent impact on the predicted listeriosis risk in all baseline conditions (e.g. presence or absence of growth inhibitor), others were greatly dependent on the initial baseline conditions or practices in the deli (e.g. preslicing of products, sanitation practices). Overall, the key findings from this scenario analysis are as follows: (i) Control growth: Using practices that prevent bacterial growth reduced the predicted risk of listeriosis, as observed in other L. monocytogenes risk assessments. The use of growth inhibitors for suitable products prevents growth of L. monocytogenes in RTE foods both at retail and during consumer home storage. In this scenario analysis, use of growth inhibitors led to a large reduction in the predicted risk of listeriosis (~95%) for all baseline conditions. The strict control of temperature during refrigerated storage in retail delis also reduced the predicted risk. The impact of temperature is nevertheless lower, as it reduces growth only during this specific storage time (5 to 20% reduction according to the baseline and the scenario). Temperature control and storage time at the consumers' homes led to a far greater risk reduction (approximately À99%).
(ii) Control cross-contamination: Eliminating crosscontamination of L. monocytogenes in the retail environment significantly reduced the predicted risk of listeriosis. Although cross-contamination during the routine operation of the retail deli is not amenable to a simple solution, effective cleaning and sanitizing practices can reduce the likelihood that contamination in the retail environment will be a source for cross-contamination.
(iii) Control contamination at its source: Decreasing the levels and transfers of L. monocytogenes from incoming products, the environment, or niches directly decreases the predicted risk of illness in all baseline conditions. Decreasing L. monocytogenes concentration in incoming product decreased the predicted risk of listeriosis whether or not the contaminated RTE product itself supported growth. The decrease in predicted risk was greater when the equivalent contamination occurred on product that supported the growth of L. monocytogenes.
(iv) Identify key routes of contamination: The slicer is a primary source of L. monocytogenes cross-contamination to deli meats and cheeses. Control of L. monocytogenes cross-contamination at this point during retail preparation reduced the predicted risk of listeriosis in all baseline conditions.
(v) Continue sanitation: Sanitation practices that eliminate L. monocytogenes from deli FCS result in a reduction in the predicted risk of illness. Cleaning and sanitizing FCS reduced the predicted L. monocytogenes levels in the deli area. Wearing gloves while serving customers reduced the estimated risk of listeriosis.
In summary, these scenario analyses and more generally this QRA, improve our understanding of the extent and prioritize the various factors that contribute to L. monocytogenes contamination in the retail deli. This work is intended to encourage improvements to retail food safety practices and mitigation strategies to reduce L. monocytogenes contamination of RTE foods.