8:28
James Webb Space Telescope reveals a universe of sights and sounds

It’s the universe as we’ve never experienced it before. The James Webb Space Telescope is sending back incredible images of deep space so advanced scientists believe it’s going to “change astronomy forever.”

It’s not only that we can see into space and time billions of years ago. The magic is that we can see anything at all.

Although its predecessor the Hubble Space Telescope offered up some incredible sights, Webb, which was developed in partnership with NASA and the Canadian and European space agencies, is able to look even further back in time and show us more detail about what lies beyond planet Earth.

Take the recent release of the Pillars of Creation which was first captured in 1995 by Hubble. In the original image from the area, which is considered to be a star-making part of the galaxy, pillars of gaseous clouds that look like long fingers are reaching up to the sky.

What we couldn’t see before, and what is now revealed by the Webb telescope, are all the stars hidden behind the gas.

That’s because Webb sees infrared light, which is ordinarily invisible to humans.

By picking up infrared light, Webb can see objects that are so far away, the light they emit takes over 13.5 billion years to reach Earth. That means Webb is also like a time machine in that it can see what the universe looked like back when the earth and sun were formed.

However, what Webb is sending back is invisible to humans because we aren’t able to see infrared light.

So it’s the job of Joe DePasquale and Alyssa Pagan, science visuals developers at the Space Telescope Science Institute in Baltimore, to translate the information from Webb into something visible.

Joe DePasquale, senior science visuals developer, creates images from the James Webb Space Telescope.

“We can’t see in the infrared. So there has to be some level of translation here. But we use physical meaning like true physical science in order to represent the colour,” Pagan told Global’s The New Reality.

With the help of NASA scientists, Pagan and DePasquale break down the images into wavelengths. “We apply colour according to those wavelengths. And so the shortest wavelength filters that we have, we use blue for those. And as we move into longer and longer wavelengths, we go to greens and then reds,” DePasquale says.

The end result is eyepopping images like the mountainous-looking cosmic cliffs of the Carina Nebula captured by Webb.

“What we’re seeing when we look at these images is the raw material for life,” DePasquale says.

“We’re understanding the universe. We’re understanding ourselves. It’s so intriguing to get this new perspective, this bigger picture. A lot of people can be like, ‘Oh, it makes me feel small,’ but I think for a lot of people it actually makes you feel unified, connected, part of something that’s so grand and so beautiful. So you are a part of something that’s awesome.”

In their own right, these images are showstoppers, yet a Canadian scientist is now adding another level of emotion to it all.

Matt Russo, a University of Toronto physicist and a sonificiation specialist, is working with musician and friend Andrew Santaguida to add sound to the universe.

“The whole process felt really natural because we’re combining things that we’re passionate about: music, astronomy, math, computer programming, science, communication — all of these things wrapped up into one bundle,” Russo says.

Matt Russo, a University of Toronto physicist and sonification specialist, creates sounds for the Webb images.

Their first effort at sonifying an image was with the Trappist-1 solar system, first captured by NASA’S Spitzer Space Telescope in 2017.

“[It] is an amazing solar system with seven earth-sized planets. But they also happened to be locked in a musical pattern called an orbital resonance. And so that made it really natural to convert their motions into musical rhythms and pitches,” Russo says.

They did the sonification of Trappist for pure enjoyment — then NASA took notice.

“We kind of just on our own, (started) sonifying different things (NASA) had released and we would send to them and they would just start posting it on their own. And then eventually that led to us working for them professionally.”

Some of the sonifications have been met with skepticism from the public, like when they did the sound for a black hole.

“There’s a real soundwave detected in space in a galaxy cluster. And we were able to see the waves in the image, which means we can extract them and re-synthesize a sound,” Russo says.

“Some outlets would say it’s an actual recorded sound of a black hole, as if you had a microphone in space, which we know would not work for several reasons. So it’s important when we do sonification to present it for exactly what it is: that it’s data converged into sound.”

Now Russo and Santaguida are working on the latest imagery from the James Webb telescope.

They’re taking the spectacular images DePasquale and Pagan have created and putting them through a software system that Russo designed.

According to Russo sometimes the sound from the data can be a pleasant surprise.  Other times they need to get a bit more creative to figure out how best to represent something in the image. Russo says they always try to be as scientifically accurate as possible.

“Where we have a little more musical input, we have to decide, for instance, which musical instrument is going to be triggered by stars,” he adds. “People seem to have an intuition that stars would make kind of a bell or chime sound.”

Their sonifications of the Webb images are now allowing people to see — and hear — the universe.

The sonifications are providing those living with visual impairments the chance to experience new insights into what’s out there.

“The whole goal is to communicate those interesting features in the image, through sound,” Russo says.

Christine Malec, a member of the visually impaired community in Toronto and an arts and culture consultant, says the sonifications by Russo and Santaguida allow her to conceptualize the images from the telescope, even though she is not able to see them.

“I had never imagined experiencing astronomy in that way,” she tells The New Reality.

Christine Malec, is a member of the visually impaired community, helping NASA make Webb images more accessible.

“When I experienced the sonification for the first time, I felt it in a way that was not intellectual; it was sensory and visceral. So I sometimes wonder if it’s what sighted people experience looking up at the night sky,” Malec says.

She now works regularly with Russo, Santaguida and NASA to help best translate the images from Webb for the benefit of people living with visual impairments.

Malec is excited about the future of space exploration and is hopeful for the future of accessible content in the science field.

“I wonder if I was a child now and came across things like sonification and image descriptions and astronomical stuff, would a career in STEM make more sense? Would it be more appealing? And I think the answer to that is yes. So I think that reason is a really good one for blind and low vision kids today to grow up with this as normal, I think it’s incredibly valuable.”