Background: While the COVID-19 outbreak in China now appears suppressed, Europe and the USA have become the epicentres, both reporting many more deaths than China. Responding to the pandemic, Sweden has taken a different approach aiming to mitigate, not suppress, community transmission, by using physical distancing without lockdowns. Here we contrast the consequences of different responses to COVID-19 within Sweden, the resulting demand for care, intensive care, the death tolls and the associated direct healthcare related costs. Methods: We used an age-stratified health-care demand extended SEIR (susceptible, exposed, infectious, recovered) compartmental model for all municipalities in Sweden, and a radiation model for describing inter-municipality mobility. The model was calibrated against data from municipalities in the Stockholm healthcare region. Results: Our scenario with moderate to strong physical distancing describes well the observed health demand and deaths in Sweden up to the end of May 2020. In this scenario, the intensive care unit (ICU) demand reaches the pre-pandemic maximum capacity just above 500 beds. In the counterfactual scenario, the ICU demand is estimated to reach ∼20 times higher than the pre-pandemic ICU capacity. The different scenarios show quite different death tolls up to 1 September, ranging from 5000 to 41 000, excluding deaths potentially caused by ICU shortage. Additionally, our statistical analysis of all causes excess mortality indicates that the number of deaths attributable to COVID-19 could be increased by 40% (95% confidence interval: 0.24, 0.57). Conclusion: The results of this study highlight the impact of different combinations of non-pharmaceutical interventions, especially moderate physical distancing in combination with more effective isolation of infectious individuals, on reducing deaths, health demands and lowering healthcare costs. In less effective mitigation scenarios, the demand on ICU beds would rapidly exceed capacity, showing the tight interconnection between the healthcare demand and physical distancing in the society. These findings have relevance for Swedish policy and response to the COVID-19 pandemic and illustrate the importance of maintaining the level of physical distancing for a longer period beyond the study period to suppress or mitigate the impacts from the pandemic.