Carlisle supported sectional compromise to keep the slave states from seceding in the winter of 1860-1861. After the Civil War began, he voted for the Kentucky legislature's proclamation of neutrality, and did not join either side's military. Since the majority of his constituents favored the Union cause, he lost a reelection bid in September 1861. He aligned himself with the Peace Democrats, who sought a negotiated settlement and restoration of the Union status quo antebellum (i.e., with slavery intact). In 1866 and again in 1869, Carlisle was elected to the state senate, where he spoke out against Radical Reconstruction. In 1871, he was elected Kentucky's lieutenant governor, allowing him to gain parliamentary experience while presiding over the state senate.

In 1876, Carlisle won election to the first of seven consecutive terms in the U.S. House of Representatives (1876-1890). He lobbied unsuccessfully for the repeal of the Specie Resumption Act of 1875, which was scheduled to return the U.S. to the gold standard in January 1879. He took a moderate bimetallist position, endorsing the use of silver as well as gold, but opposing the inflationist policy of the unlimited coinage of silver (free silver). When the Democrats won control of the House in 1878, Carlisle's outspoken support of the Democratic attempt to roll back the civil rights legislation of Reconstruction earned him respect among his partisan colleagues, although Republican president Rutherford B. Hayes vetoed the bills.

As a member of the powerful House Ways and Means Committee, Carlisle pushed for tariff reduction, arguing that high tariffs helped only special business interests to the detriment of farmers, workers, and consumers. During the acrimonious debate in Congress which resulted in passage of the Mongrel Tariff of 1883 (so named because it was a compromise which did not satisfy either side), Carlisle emerged as a leader of the low-tariff/free trade Democrats. In December 1883, he won the House speakership over trade-protectionist and former speaker Samuel J. Randall of Pennsylvania, and was reelected in 1885 and 1887.

In 1884, Carlisle was Kentucky's favorite-son candidate for president, but lost the nomination to Grover Cleveland of New York. After President Cleveland appealed for lower tariffs in his annual message of December 1887, Speaker Carlisle redoubled his efforts to pass the reform. The result was the Mills Bill of 1888, which failed in the Republican-controlled Senate. When the Republican won the White House and both houses of Congress in the 1888 elections, tariff reformers were in the minority on Capitol Hill.

As minority leader, Carlisle vigorously opposed the rules imposed by the new, Republican speaker of the house, Thomas B. Reed of Maine, which enhanced the speaker's authority to halt dilatory practices. The timing of his election to the U.S. Senate in May 1890 to fill a seat vacated by the death of Senator James Beck, allowed Carlisle to vote against the protectionist McKinley Tariff in both houses (the bill passed both). He also voted against Lodge Federal Elections Bill and the expansion of veterans' pensions.

When Cleveland was reelected in 1892, he appointed Carlisle as treasury secretary. The slowing economy of 1892, under the watch of President Benjamin Harrison, grew into a full-fledged depression shortly after the Cleveland administration took office. Carlisle and Cleveland lobbied for Congress to repeal the Sherman Silver Purchase Act of 1890 during a special session in 1893. It passed, but the debate further divided the hard money (gold standard) and soft money (free silver) wings of the Democratic party. The Treasury Departments sale of bonds to J. P. Morgan's banking syndicate provoked heated criticism. When the Democrats nominated free-silver champion William Jennings Bryan of Nebraska for president in 1896, Carlisle backed the ticket of the remnant Gold Democrats.