The mosque was commissioned by Ahmad ibn Ţūlūn, the Turkic Abbassid governor of Egypt from 868–884 whose rule was characterized by de facto independence. The historian al-Maqrizi lists the mosque's construction start date as 876 AD, and the mosque's original inscription slab identifies the date of completion as 265 AH, or 879 AD.
The mosque was constructed on a small hill called Gebel Yashkur, "The Hill of Thanksgiving." One local legend says that it is here that Noah's Ark came to rest after the Deluge, instead of at Mount Ararat.
The grand ceremonial mosque was intended as the focal point of Ibn Ţūlūn's capital, al-Qata'i, which served as the center of administration for the Tulunid dynasty. The mosque originally was backed by ibn Ţūlūn's palace, and a door adjacent to the minbar allowed him direct entry to the mosque. Al-Qata'i was razed in the early 10th century, AD, and the mosque is the only surviving structure. The mosque was constructed in the Samarran style common with Abbasid constructions. The mosque is constructed around a courtyard, with one covered hall on each of the four sides, the largest being on the side of the qibla, or direction to Mecca. The original mosque had its ablution fountain (sabil) in the area between the inner and outer walls. A distinctive sabil with a high drum dome was added in the central courtyard at the end of the thirteenth century by the Sultan Lajīn.
There is significant controversy over the date of construction of the minaret, which features a helical outer staircase similar to that of the famous minaret in Samarra. It is also told that using these stairs one can climb up on a horse. Legend has it that ibn Ţūlūn himself was accidentally responsible for the design of the structure: supposedly while sitting with his officials, he absentmindedly wound a piece of parchment around his finger. When someone asked him what he was doing, he responded, embarrassed, that he was designing his minaret. Many of the architectural features, however, point to a later construction, in particular the way in which the minaret does not connect well with the main mosque structure, something that would have been averted had the minaret and mosque been built at the same time. Architectural historian Doris Behrens-Abouseif asserts that Sultan Lajīn, who restored the mosque in 1296, was responsible for the construction of the current minaret.
The mosque has been restored several times. The first known restoration was in 1177 under orders of the Fatimid wazir Badr al-Jamālī, who left a second inscription slab on the mosque, which is noted for containing the Shī'ī version of the shahada, adding the phrase "And Ali is the wali of God" after acknowledging the oneness of God and the prophethood of Muhammad. Sultan Lajīn's restoration of 1296 added several improvements. The mosque was most recently restored by the Egyptian Supreme Council of Antiquities in 2004.
During the medieval period, several houses were built up against the outside walls of the mosque. Most were demolished in 1928 by the Committee for the Conservation of Arab Monuments, however, two of the oldest and best-preserved homes were left intact. The "house of the Cretan woman" (Bayt al-Kritliyya) and the Beit Amna bint Salim, were originally two separate structures, but a bridge at the third floor level was added at some point, combining them into a single structure. The house, accessible through the outer walls of the mosque, is open to the public as the Gayer-Anderson Museum, named after the British general R.G. 'John' Gayer-Anderson, who lived there until 1942.
Parts of the James Bond film The Spy Who Loved Me were filmed at the Mosque of Ibn Tulun and in the Gayer-Anderson Museum. The mosque is also featured in the game Serious Sam 3: BFE, forming a significant part of the game's third level.