The museum was originally established in 1961 through the generosity of Amon G. Carter Sr. (1879–1955), to house his collection of paintings and sculpture by Frederic Remington and Charles M. Russell. The museum continues to collect and exhibit masterworks of American art.
The collection is housed in a landmark international style building designed by New York architect Philip Johnson, blending modern materials with classical design. Johnson created a simple, elegant building that combines the warmth and richness of bronze with the creamy, intricately patterned surface of native Texas shellstone. His design was inspired by the Renaissance-style loggia---a covered, open gallery that looks out over an open court. Johnson has been involved in each subsequent expansion.