Replay Technologies’ ‘freeD’ To Deliver Real-Time 3D Sports Replays To Mobile Devices

In an age of video on demand (VOD), sporting events are among the few broadcasts still watched live. More than one billion people are thought to have watched the 2014 World Cup Final and in the US alone, 100 million viewers watched the 2015 Super Bowl.

Such is the reach of Israeli startup Replay Technologies. Founded five years ago, the company has developed a method of filming called freeD, which generates instant, real-time 3D replays that have already been used at the 2012 London Olympics, in Yankee Stadium, and just last week, at Intel’s demo at the Consumer Electronics Show (CES) in Las Vegas.

SEE ALSO: CES 2016: The Best Of Israeli Tech

“With this technology, you’ll be able to view the game from any angle you want. You will become the director,” Intel CEO Brian Krzanich, who spoke at CES, said right before showing Replay’s shot of LeBron James’ slam dunk. The chip giant is partnering with the Israeli startup to deliver 3D sports replays not only in stadiums and on broadcast (which Replay Technologies did up until now), but also on one’s own iPad.

With Intel, the company is developing viewer-controlled imaging, so that from smartphones, tablets and gaming consoles, fans will be able to zoom in and pan out, change perspectives, and pivot orientation. “You’ll be able to walk through the field with the players,” says Jonathan Levene, Replay’s VP of corporate development. ‘It’s going to be your game.’

Such is the gospel of the new media age. According to a recent Nielsen report, American adults consume daily between two to three hours of video on computers, smartphones, and gaming consoles, a trend that is changing media content itself. As Jeff Stier and Chris Gianutsos, advisers at Ernst & Young, told Wired: “With the evolution of the omni-platform environment, television storytelling can be splashed across multiple screens simultaneously… Viewers increasingly want to be a part of the experience.”

No goofy 3D glasses are needed

Using up to 32 small, high-resolution cameras installed at the rim of the sporting arena, Replay captures a three-dimensional image of the field and the players on it. “We know where every pixel is in space and can recreate any viewing angle,” Levene explains.

Yet, Replay Technologies doesn’t use the flat, two-dimensional pixels of a standard camera. Rather, the company maps the scene with volumetric pixels (volume is what regular pixels don’t have) called ‘voxels,’ which hold data on three vectors – length, width, and depth. Says Levene: “It looks like 3D because it is 3D” – no goofy glasses needed.

According to the company’s creative director Diego Prilusky, “you can position the camera straight in the middle of the field, looking at the face of the player. That is something that you cannot do during the game but now we can really freeze the moment, and travel around from front and behind and really see what’s going on from the eyes of the player and how he perceives the game.”

From an on-site production booth, Replay Technologies can compile clips from all their camera feeds and compress what is easily a terabyte of data in less than 60 seconds. Their next-generation technology is expected to cut that processing time in half.

SEE ALSO: Intel’s 3D Vision Technology Enables Drones To ‘See’

Levene says that the technology is “galaxies apart” from what’s currently used. Goal-line technology, which is used in tennis, football and soccer to track the path of the ball, amalgamates multiple two-dimensional video frames to recreate the trajectory of the ball. However, Levene stresses it’s not just the 3D special effects that have garnered repeated attention. Rather, it’s the enhanced storytelling, especially at controversial moments. “Did the runner cross the plate before the catcher tagged it? In one shot we can tell that story,” he says.

The sports announcers love freeD; in the Champions League Finals, German media company Sky Deutschland used nine clips per game.

Soccer via Biser Todorov/WikiCommons

After raising $9 million in private financing rounds, Replay – led by co-founder and CEO Oren Haimovitch-Yogev – went on to sign multi-year agreements with American teams, such as the New York Yankees, the Dallas Cowboys, and the Cleveland Cavaliers. They have also installed cameras in the Maracanã Stadium in Rio de Janeiro after signing a deal with TV Globo, and soon, their cameras will be in Asia.

But most importantly, freeD will soon be in the hands of fans!

Photos and videos: Replay Technologies, Biser Todorov, Cliff