The fatigue damage is one of the most common distresses observed on the asphalt concrete pavement. The initiation and propagation of the fatigue damage is a complicated phenomenon and very difficult to detect. In order to thoroughly understand the fatigue of asphalt concrete, the behaviors of the key components of asphalt concrete under cyclic loading are investigated respectively. A new experiment method is developed to test the performances of asphalt binder, mastic and mixture under cyclic loading, which provides a tool to unify the fatigue test method for both binding medium and asphalt mixture. Using the new fatigue test method, the effects of loading magnitude, temperature and loading rate to the performance of the asphalt binder under cyclic loading are estimated. Mastic and mixture specimens are prepared by adding fillers and controlled-size aggregates into the asphalt binder. The effects of filler content to the performance of mastic specimen are discussed. The differences between the test results of mastic and mixture are compared and analyzed. Incorporated with the new fatigue test, x-ray tomography system is used in this study to: 1. Analyze the structure change of the mastic specimen before and after the fatigue test. 2. Compare the void content differences between the mastic and mixture specimens. 3. Reconstruct the 3-D internal structures of mastic and mixture specimens to build up the digital specimens. The digital specimens are used in the fatigue simulation of the asphalt binder, mastic and mixture specimens based on the finite element method. The asphalt binder, filler and aggregate are treated as different materials. Damage parameter is introduced to model the degradation of elastic modulus of the asphalt binder caused by fatigue damage. Direct cyclic analysis available in ABAQUS is used to obtain the response of the material after large number of loading cycles. The basalt fibers are dispersed into the asphalt binder and mastic specimens, the effects of the basalt fiber to the performances of the binder and mastic at low temperature are analyzed using both experimental and FEM modeling methods.