The castle stands on the highest point of a promontory that serves as a natural defence, with steep cliffs to the north and east. The site extends to the west along a relatively broad and flat spur, from which the gate tower is accessed.

The earliest documented occupation is from the Almohad era, between the 12th and 13th centuries. This period saw the construction of the fortress, whch is divided into three enclosures equipped with towers in the shape of truncated pyramids. These enclosures are laid out in tiers from north to south and were built from high-grade rammed earth, with lime mortar decorated to resemble stonework. During the Muslim era the hamlet lay to the south, on the banks of a thalweg. 

Oer the course of the 14th century, La Iruela underwent a series of modifications, including the construction of its first church and a wall surrounding the area providing access to the castle and hamlet. Various repairs were made to the castle at this time and what is known as the Picacho (“summit”) Tower was erected at the top of the crag.

During the 15th century the entrance to the fortress, which was located in the wall separating the town from the bailey, was replaced by a gate tower that still provides access to the castle today. Rectangular in shape, its lower section is slightly wider than the rest of the structure in order to lend it greater solidity. It is also the only remaining part of the original gate tower, as the upper sections have undergone thorough restoration.

Over the course of the following century, the Church of Santo Domingo was built on the castle’s south flank. Designed with a typically Renaissance hall-church layout, it has three naves (with a double-width central nave) and a flat headwall. There are two circular towers at the corners of the chevet. In the 19th century, as the church fell into a state of disrepair, it was no longer used as a place of worship and instead served as the municipal cemetery until the middle of the next century.

The origins of the town of La Iruela are seemingly linked to the construction of the Almohad castle, which was built on a rocky spur overlooking the settlement. It is very likely that the complex was built as the hub of a military division encompassing much of what would later become the Adelantamiento de Cazorla. The high quality of the materials and execution suggests that the central government was involved in the castle’s construction, with the aim of strengthening the borders of al-Andalus. 

Before he conquered the area in 1231, Ferdinand III granted the villa of Quesada as a feudal estate to the Church of Santa María de Toledo. Some months later, the Primate Archbishop of Toledo, Rodrigo Ximénez de Rada, began the process of conquering this territory (including La Iruela) and thereby seized control of the lands that would subsequently comprise the feudal estate of the Adelantamiento de Cazorla. The choice of Cazorla as the capital of the Adelantamiento had a detrimental effect on the development of the nearby towns, and after the construction of Cazorla’s own fortifications, La Iruela Castle was all but abandoned.

In the second half of the 14th century, when civil war broke out between King Peter I and his half-brother Henry II of Trastámara, Cazorla remained loyal to the king while La Iruela supported the Archbishop of Toledo, who was on the side of the rebels. When Henry’s victory brought the conflict to an end, in 1378 Archbishop Pedro Tenorio granted the town the status of villa in gratitude for the support it had offered.

From that moment on, La Iruela began a process of gradual development, in which new buildings were constructed and its defensive structures were repaired and strengthened. After the conquest of Granada, the Adelantamiento lost its strategic importance as a border region and no longer had a military governor, even though it continued to exist until 1812. 

Over the course of the 16th century, the population of La Iruela tripled, resulting in the construction of the town’s first new district and the demolition of part of the walled enclosure that had surrounded the settlement since the Middle Ages.

During the Peninsular War the town was attacked on a number of occasions by French troops, suffering significant damage as a result.