Your central railroad terminal is your most accessible point in your network, because it connects to the largest number of routes. So there is a natural temptation to want to put it in the most central point, where people can walk to the largest number of activity generating buildings.
By that logic, we should have a freeway interchange in the middle of every CBD. In reality, the freeway interchange always winds up at the fringe of the CBD - still providing excellent access, but not permitted to displace the core. In a private market, they can't afford the real estate. Worse yet is when your Central station is also a terminal station, where routes end*. When that happens, large amounts of space are needed to store vehicles--and long trains require long storage tracks and long access tracks, so you are basically devoting an entire series of blocks. And that happens even when that space is very expensive. Both the Manhattan and Washington DC have enormous 'trail track' complexes, despite the staggering value of land those complexes represent. (The Hudson Yards development is, guess what, skyscrapers build over/between the train tracks of the railway's Hudson Yard).
Union Stations, a central depot for all the railroads serving a city, is a peculiarly American obsession. I suspect part of it was city boosterism (as cities competed to produce the most beautiful civic megaproject) and the other half was clearing real estate for urban development. American cities of a certain age have amazing city halls, courthouses, and railroad terminals (veritable 'Cathedrals of Transportation'). Many will object that the alternate is the mess under Madison Square Garden. And the aforesaid urge still exists, as Calatrava's World Trade Center Transportation Hub and the Moynihan Train Hall clearly attest.
But from a network design perspective, from a land use perspective, we are better not having a "Central Terminal"--we are better having a central station with through running, where lots of trains stop, but none of them are stored waiting.
There are bogus arguments made about the need to store them there for peak commuters. The peak was never the whole story, and post-pandemic represents and even smaller fraction of the total. (Peak hour design is only relevant for assessing capacity restraints). If a train can be stored at a central terminal, a train can also be stored at a peripheral terminal 15 minutes away and scheduled to arrive at the proper time, while also providing additional service to nearby stations along the way.
Some of the limitations to through running are infrastructural (Oh, Chicago, I'm so sorry). But many are merely operational. NYC is a notorious offender, because the different competing public agencies can't manage the necessary operational coordination necessary to share track. But the same issues exists at DC's Union station, where the Virginia Railway Express (VRE) and Maryland-Area Regional Commuter (MARC) railways meet. (Admittedly, efforts to resolve the issue are in-process).
This may seem like a rail issue, specific to only a few large metros, but the same issues afflict smaller bus-based transit systems across America--Raleigh is afflicted by it, Des Moines is afflicted by it, etc. Trying to fit enough space to story buses into a downtown area means the terminal must be peripheral, which means it is less effective, and also more intrusive, more disruptive to walkable urbanism.
*Worst of all is when some genius attempts to maximize management convenience by also centralizing maintenance depot operations in the same complex.