In the decades after the Civil War the northward expansion of the city accelerated greatly. The large department stories began to abandon stores south of 14th and rebuilt — larger and more elaborate stores — in an area bounded by 14th and 23rd Streets, Broadway and Sixth Avenue (the area that came to be known as the Ladies’ Mile. The theaters began to migrate as well, moving north from Union Square. Many established themselves in the blocks between West 28th and 31st Streets.
West of Sixth Avenue the side streets began to shift from residential to entertainment of the more notorious type: saloons, gambling and dance halls, fight clubs, and many dozens of brothels with names like Star and Garter, Buckingham Palace, the Bohemia, and the Tivoli. In Low Life, author Luc Sante breaks down the area: 29th Street was where to find prostitution; 28th, high stakes gambling; 27th, low stakes.
This area became known as The Tenderloin or Satan’s Circus (how these names were coined had to do with corrupt police and moral reformers, stories often told, so for example by Ephemeral New York.)
One of the most famous (and infamous) saloons in the Tenderloin was the Haymarket. The entrance was south of 30th Street on the east side of Sixth Avenue. You could get on the Sixth Avenue El and travel downtown right to the Jefferson Market police court, if your luck ran out.
The Bowery Boys have a description which is the is the most concise and clear I’ve come across:
Haymarket was the Tenderloin jewel, a three-story dance hall illuminated (disguised?) like a legitimate Broadway theatre and named for an even more legitimate British theater district. New York’s chief of police in 1887 described it as “animate with the licentious life of the avenue.”
Manhattan Unlocked has an excellent and very detailed post about the history of the Tenderloin, which includes this map:
The short gray line at the bottom shows the southern boundary of the Tenderloin on 23rd Street (below it was Chelsea to the left with the Ladies’ Mile Shopping district directly beneath). The extensive purple lines at the left show Hell’s Kitchen, bordering 8th Avenue (much of it is considered Chelsea today). There was a viable African American district, indicated by the green lines, that straddled 7th Avenue, half in/half out of the Tenderloin. This was the African American community that staked out Harlem in the 1910s when Penn Station was built and they had to re-locate. The blue lines show the heaviest concentration of brothels, saloons and gambling halls, all west of Broadway. The yellow lines show Fifth Avenue and Broadway, which were ritzy streets of theaters, fine hotels and shopping. The single, slanted red line was the Haymarket—smack in the middle. The Gilsey House and the Breslin (Ace) Hotels are the pink dots, on the east side of Broadway.