The first library in Jeffersontown opened May 16, 1911, after the ladies of Jeffersontown's Current Events Club solicited contributions of money and books from the community. The library was free and open to everyone in the county. Fannie Belle Snively volunteered her time as the first librarian until a more permanent arrangement could be made.

The library opened on the second floor of the Bruce Building, but tended to move around to various homes and buildings on or near the town square. In 1939, the library operated out of the second floor of the Municipal Building, which also housed the fire department and city hall. Several other locations, including the Jeffersontown High School, housed the library until 1947, when Betty Elswick opened a library on the back porch of her home on Maple Road. The library grew from there, moving to the annex of the Presbyterian Church and then to the J-town Shopping Center in 1962, when the library was officially established as part of the Louisville Free Public Library.

In 1970, the library relocated to Colonial Manor on Watterson Trail, the original site of the County Poor Farm. One final move was made, when the Jeffersontown Library Branch and Museum held its grand opening on January 29, 1996, in a newly constructed building next to Colonial Manor. The Jeffersontown Library continues to serve as one of the most popular branches in the Louisville system.