End-to-end 100-TOPS/W Inference With Analog In-Memory Computing: Are We There Yet? (Full Report)