To enhance the resource utilization of shield tunnel spoil and improve the mechanical properties of recycled concrete while investigating its failure mechanism, this study designed four concrete mix proportions: without coarse aggregate (NCS), single-sized aggregate (1C10), double-sized aggregate (2C15), and well-graded aggregate (3C20). Uniaxial compression tests and Digital Image Correlation (DIC) techniques were employed to analyze the strength development and failure mechanisms at 28-day and 56-day curing ages. The results shows: 1) The NCS group achieved a 28-day compressive strength of 54.19 MPa, meeting the C50 design strength requirement. However, its 56-day strength development was limited, exhibiting the largest crack width and significant brittle failure characteristics. 2) The 1C10 group achieved the highest 56-day compressive strength (79.76 MPa), with tortuous crack propagation and a multi-crack ductile failure mode. 3) The 2C15 group demonstrated the optimal compressive toughness index, indicating excellent toughness and energy absorption capacity. 4) The 3C20 group showed significant strength regression (43.3 MPa), failing to meet the design strength requirement, with cracks penetrating aggregates and exhibiting obvious brittle failure. The study concludes that while the NCS group meets the basic strength requirement, adopting single-sized or double-sized aggregate gradations can effectively optimize crack paths and failure modes. It is recommended to control the maximum aggregate size not exceeding 15 mm in practical applications to ensure the strength and durability requirements of shield tunnel spoil concrete.