He is known as one of the masters of the art world, with an inimitable style that has captured audiences the world over, but now Rembrandt has a rival: artificial intelligence. 

A machine-learning algorithm has been used to solve the mystery of what Rembrandt’s famous masterpiece The Night Watch originally looked like almost four centuries ago, by recreating four missing sections and allowing viewers to appreciate the complete painting again for the first time.

The noted artwork depicting the civic guard of 1642 is currently 363cm by 438cm but was originally even larger. 

When the painting was moved to Amsterdam’s city hall in 1715, it didn’t fit through the doors – so it was trimmed on all four sides, removing vital details and changing the composition. 

These discarded strips have never been found, although a small copy from the time, attributed to Gerrit Lundens and owned by London’s National Gallery, suggests what they might have looked like.

So scientists at Amsterdam’s Rijksmuseum trained three sets of computer algorithms modelled on human brains – known as artificial neural networks – to recognise the differences between the original and the copy and fill in the blanks, Rembrandt-style.

The reconstructed painting, with four extra section on each of its sides

Credit: Peter Dejong

… and the trimmed version of the painting that has been on display for centuries


The result is the world’s first ever reconstruction of the complete masterpiece.  

“It is wonderful to be able to now see with our own eyes The Night Watch as Rembrandt intended it to be seen,” said Rijksmuseum director Taco Dibbits.

“Rembrandt was asked to make a group portrait, but what did he do? He painted a story, bringing dynamics and movement. In the reconstruction, this is crystal clear.”

By studying the two versions of the painting, the algorithms were able to reproduce Rembrandt’s original down to a tee, including the perspective, style and colour palette. 

For added authenticity, the machine even looked at how the painting had aged over the years, so the recreated strips are complete with small cracks.

A new section of 'The Night Watch'


“As far as I know, this is the first time this has been done,” Robert Erdmann, senior scientist at the Rijksmuseum, told The Telegraph. 

“Neural network technology has seen an explosion in recent years. Most of the innovation happens in Silicon Valley. It’s unusual for a museum to be at the forefront of this research.”

“It was quite a challenge. The copy was different in many ways. It is a fifth of the size, it’s by a different painter and in a different style, and it has a different palette.”

Museum workers check panels of reconstructed Rembrandt's masterpiece 'The Night Watch' before assembly in the Rijksmuseum

Credit: REMKO DE WAAL/EPA-EFE/Shutterstock

The final, computer-generated images were printed onto canvas, varnished, and mounted upon metal supports very slightly in front of Rembrandt’s painting to give a visual illusion of the original work.

“It really is the case that the cut-off pieces have an influence on the painting’s composition,’ said Mr Dibbits. “In the original picture, the captain and lieutenant were just off-centre.”

The sense of space at the top of the painting, he said, allows it to “breathe”, while shadows at the bottom increase the sense of perspective and movement. A balustrade on the left makes it clear the guardsmen are on a bridge.

“Whoever snipped off pieces clearly thought about it a lot, decided [the protagonists] should be in the middle and neatly snipped off some bystanders. Now, more dynamic has returned to the painting, they are walking forward, the troops are beginning to march.”