The old Penn Station was partially torn down in the 1960s, with the tracks surviving as they are today underneath Madison Square Garden. The part of Penn Station above ground level was torn down.
https://en.wikipedia.org/wiki/Pennsylvania_Station_(1910%E2%80%931963)
![](https://upload.wikimedia.org/wikipedia/commons/thumb/b/ba/Penn_Station3.jpg/240px-Penn_Station3.jpg)
The Post Office was a separate building, but of the same style and by the same firm. The old Post Office has railroad tracks underneath it has provided some people with the idea to use it instead of the present Penn Station (and its original name was also "Penn Station" since it was just across the street from Penn Station):
https://en.wikipedia.org/wiki/James_A._Farley_Post_Office_Building
![](https://upload.wikimedia.org/wikipedia/commons/thumb/f/f8/James_Farley_Post_Office_c1912.jpg/220px-James_Farley_Post_Office_c1912.jpg)
Some work on the Post Office to be a new "Penn Station" was started in 2010 to be finished in 2016:
https://en.wikipedia.org/wiki/Pennsylvania_Station_(New_York_City)#Moynihan_Station
http://www.nj.com/traffic/index.ssf/2016/01/cuomo_kick_starts_moynihan_station_plan_to_free_sp.html#incart_river_home
The plan is to still use the present Penn Station, whose tracks and concourse survives, with some changes, from the original Penn Traffic building and extending the present station to the old Post Office.
This appears to be "Doable" but only because the tracks and concourse survives from the old Penn Station. The plan is to have the stop at the old Post Office and use the concourse from the old Penn Station. The Post Office stopped using rail to move mail in the 1960s (Through some mail was moved on the Pennsylvania train till the 1990s) but the rail lines ran underneath the Old Post Office, which also has sidings for rail cars.
The problem is the Old Post Office was NEVER designed to handle a large number of people, unlike the Old Penn Station. Thus use of the tracks and concourse of the old and new Penn Station have to be retained and used to handle the volume of people that are expected (If the Feds and locals ever do build that tunnel everyone said was needed in the 1960s).
Using the old Post Office is NOT the ideal solution (Rebuilding Penn Station would be the best solution) but given what CAN be done, the best option we can expect at present.