Structural risk-based evaluation requires a large number of time-history analyses at different ground motion (GM) intensity levels, where the scenarios (e.g. magnitude and distance) of the GMs used in the time-history analyses should be consistent with the site's hazard. The current practice of GM selection typically simplifies the choice of scenario to either an average scenario or the modal scenarios based on the site's hazard deaggregation results. This paper investigates the impact of hazard deaggregation and scenario selection on estimating structural seismic risk. For a hypothetical site in the Eastern US, a Monte Carlo seismic hazard analysis is performed to derive a site-consistent GM suite that captures 1,000,000 years of the site's seismic activity. The complete GM suite consisting of 99,917 records is then used to perform nonlinear dynamic analyses on a mid-rise concrete office building to derive a benchmark seismic demand curve. Subsequently, four GM sets are selected based on average and modal scenarios from two different hazard deaggregation formulation, and the resulting demand curves are compared to the benchmark. The results show that the hazard deaggregation method and scenario choice impacts the demand curve estimation. When deaggregation is performed on IM exceedance, GMs that were selected based on both methods agree well with the benchmark up to higher damage states where mode-based records outperform average-based records. On the other hand, when deaggregation is formulated based on IM occurrence, average scenario-based GMs better match the benchmark, except for higher damage states where again modal scenario-based GMs are in better agreement with the benchmark.