Shale formations as major unconventional energy resources are crucial in satisfying the global energy needs of the future. Via nanoindentation method and upscale method, the macromechanical parameters of shale, such as hardness, elastic modulus, are obtained. The conventional Mori–Tanaka upscale method only divides the data into three mineral classes and fails to fully incorporate micromechanical properties to reflect the macroscale properties of samples. The research measures micromechanical parameters of shale via nanoindentation and performs cluster analysis of nanoindentation measurements. The results of cluster analysis are then combined with the Mori–Tanaka upscale model to evaluate the macroscale mechanical property of shale. The elastic modulus, hardness, and fracture toughness are divided into five groups (clusters) via cluster analysis, with each representing a certain mineral composition. This research is of great significance for more reasonably and accurately characterizing shale mechanical properties, optimizing the recovery scheme, and improving the recovery efficiency of shale gas.