In this study, numerical simulations are conducted to investigate the effects of pin fin and dimple shape on the flow structure and heat transfer characteristics in a rectangular channel. The studied shapes for dimple and pin fin are circular, spanwise-elliptical, and streamwise-elliptical, respectively. The flow structure, friction factor, and heat transfer performance are obtained and analyzed with Reynolds number ranging from 10,000 to 50,000. Channel with circular pin fin and dimple is chosen as the Baseline. Channels with spanwise-elliptical pin fins have the best heat transfer augmentation, while also accompanied with the largest friction factor. Spanwise-elliptical pin fin generates the strongest horseshoe vortex which is responsible for the best heat transfer augmentation. Besides, channels with streamwise-elliptical pin fins show the worst heat transfer augmentation and the smallest friction factors. Dimple plays an important role in improving the heat transfer. Spanwise-elliptical dimple yields the best heat transfer augmentation which is attributed to the strongest counter-rotating vortex, while streamwise-elliptical dimple shows the worst heat transfer enhancement.