The current condition of the active infrastructure speaks more to maintenance than alignment.
It's actually an east-west route, particularly east of Peterborough, but that's ok. According to 'railway folks', when the original proposal was 'high frequency' vs 'high speed', most of the alignment was suitable for decent average speeds. The stretch between (ish) Kaladar and Perth were problematic because of the alignment through the Frontenac Arch (southern extension of the Shield).
Best speeds on the current CN lakeshore corridor can be in the range of 160 kmh.. The big problem is not speed, it's conflict with freight traffic. Once you get into 'high-speed', in the vicinity of 300kph, you get into a track classification that doesn't exist in NA. Curvatures become huge and things like at-grade crossings, cutting through towns, etc. simply can't exist.
I mis-typed. I meant to say 'without' the high speed component.
Getting through Montreal, either via the north or south shore, is seen to be a problem but I don't know enough about that area.