Remote Controllers: The NEP Andrews Production Hubs
Game-changer, brave new world, paradigm shift – all could describe NEP Australia’s IP-enabled Sydney- and Melbourne-based centralised remote production centres which serve as hubs for the most advanced remote media production operation in Australia, if not the world.
As they arrive at the coal-face of cutting edge of media production, staff and clients pass a wall display honouring the late Keith Andrews, the man after whom the Sydney and Melbourne Andrews production hubs were named.
Mr Andrews passed away suddenly in April 2017. He held the positions of CEO of NEP Australia, followed by President and Chief Operating Officer of the global NEP Group. Text accompanying his portrait in the Hub describes Keith as having “… had a passion for people and those that could deliver innovation. None more so than the innovation contained within these walls.”
The 12 months following the 2017 NABShow saw that innovation move into high gear. At showtime, the SMPTE ST 2110 suite of standards for Professional Media Over Managed IP Networks was in final draft, but just one month later, May 22nd, NEP Australia had a 2110 test environment running in conjunction with Sony and a small number of manufacturers. It was the only interoperability test outside of a standards body test site.
According to NEP Australia’s Director of Technical Services, Marc Segar, “I don’t think I’m talking out of school when I say there are lots of manufacturers that this time last year didn’t have 2110, didn’t have IP boards, didn’t have an IP roadmap.
“I’d like to think that this project has helped bring some of that forward, that some of the manufacturers started to bring forward 2110 development to be able to get this project going.”
Also key to kick-starting the project was Australian telecommunications company Telstra, which provided the continent-wide connectivity required for the project (see story page 44), and long-time client and partner Fox Sports Australia.
The Sydney Andrews Hub is divided into two “hemispheres”, each equipped with two control rooms and associated support rooms. This allows NEP to keep different (and potentially competitive) productions separate while they use the facility at the same time. Staff use programmable swipe cards to access authorised areas.
Signals from the control rooms pass through the Network Operation Centre – NOC (pictured) which only NEP staff can access. The NOC also features LCD glass windows which can be set to obscure content from unauthorised view.
Marc Segar describes the NOC as “… a fusion between broadcast and network operations. We have no video cable in here.
“There’s two traditional broadcast monitoring positions, one on the left and one on the far right. So, the front bench and in the middle is our network monitoring, okay? For most projects, we’ve got two or three engineers in here, we always have a network engineer.
“There’s two master controller-type positions, one network position in the middle, so one guy here and one guy in Melbourne, of course; both monitoring the same network because we are now connected to Melbourne.
“Again, we’re a network more than we are a broadcaster. What’s that’s done is that’s introduced new roles. We have broadcast network engineers now on staff which we didn’t have before.
“I should also mention that the two studios, there are two TV studios, broadcast studios, here. They’re also connected to the hub as if they were venues. So, we no longer need to pull trucks up out the front like we used to, all that sort of stuff; we basically use the control rooms that are in here.”
Also on the NOC backbench are remote control positions for audio and audio comms.
“That’s no longer in-field,” says Marc Segar. “That’s now here because we remote control all the comms, and we remote control all the audio from here. We basically have complete remote control of the truck no matter where it is.
“This comes down to the zero latency for the talent on-site. That’s why we do that. They’re very perceptible to a few milliseconds. If they have real-time audio, there’s no reason to complain.”
If an outside broadcast truck could loosen its belt, not only would it breathe easier, its interior would look a lot like an Andrews Hub control room.
“We’re trying to make clients feel really comfortable,” says Segar. “We’re trying to give them as much flexibility, and the flexibility they don’t have in a fixed truck, by coming in here.
“All four rooms are identical. The only difference is the colour schemes in the room, everything else is the same. Benches are the same, positions are the same, monitors are the same, monitor wall is the same. It doesn’t matter which room you’re in: if you’re a client, you get exactly the same facility. The only thing you need to book is how many EVS channels you’re using, everything else is identical to each other.”
Part of the Hub control room flexibility is a moveable vision mixer panel and moveable monitor wall – the whole monitor wall.
“If you have a director/switcher, one person that’s also mixing the show, they can move the vision mixer panel, and of course can move the monitor wall. By doing that, you always have that really core sweet spot and so you can always see your programme preview.
“What we’ve done recently is decided to make [control room 2] an A-League room, and [control room 1] the NRL room – for now. So, they don’t even change configuration of vision switchers or anything. The room stays an A-League room – for now. It doesn’t have to, we can recall it under VSM, we can recall it in the switcher configuration if we want to.”
To support its IP-based production, NEP Australia acquired 48 Sony HDC-4300 high frame rate cameras, along with four sets of XVS-6000 and six sets of XVS-8000 switchers, but one notable absence from the Andrews Hub control rooms is that of camera control.
“People ask me quite often, what are we doing with CCU, where are the CCU guys?” says Marc Segar. “At the moment, they’re in the truck because those guys are technical guys for us. They rig all the point-of-view cameras, they do the commentary box, and if we didn’t have them on-site those things wouldn’t get done. But, in the future, we don’t know whether we’re still going to be doing that – we don’t know where the commentators are going to be, we have no idea whether we’re going to put full-time point of view cameras in and we never have to go and rig them again, which could happen.
“So, we’ve created a room that’s a central CCU room, camera shading room. It has two positions: one’s a 16-camera position which mimics what we do in the truck, and one that’s eight cameras which normally mimics what we do when we do a studio show here. So, eventually we could do an OB and a studio show, or two studio shows, or some hybrid of all those things in this space.”
Each production room is complemented by a separate audio room. These identical rooms are each built around a Lawo MC96 console. Flexible design philosophy allows for equipment to be moved to suit operators/projects.
“Monitoring in here is all off the same core that everywhere else has, of course, we don’t care what they monitor, and in terms of audio it’s all 5.1 cables,” says Marc Segar. “We have created a sound field in here for surround. It’s not completely correct for Dolby, because when you’re just doing a 5.1 show for sport, it’s not as critical. There are four plates everywhere in each room, and there are speaker stands and you can put them exactly where you need them for a proper sound field.
“We’ve got audio suppression all over the ceiling and over the back, this thing – these are different in each room – is to capture audio so that it is as near perfect acoustically as you can get. A lot of work that’s happened in this space to make sure it’s acoustically correct. We have also options for surround sound.”
In addition to the NOC, four control and audio rooms, the Andrews Sydney Hub features a number of “multi-function rooms”, including two small studio spaces.
“The reason we’ve built these is we don’t centralise commentary at the moment,” says Segar. “That’s all done on-site. But, there’s no reason why commentators can’t be anywhere in the hub. We don’t really care where they are. We’ve built two rooms that are slightly bigger, they’re 30m2, they’ve got a lighting grid and power and DMX, etc., up the top. If we do some augmented reality, a small set, extension, you could make this space look like an arena.”
The other multi-function rooms – two seven-person and two four-person rooms – can be allocated to CCU, overflow EVS, graphics, or other functions as needed.
CENTRAL APPARATUS ROOM
Like any central apparatus room in any traditional broadcast facility, the CAR at the Andrews Hub hosts racks of devices, whirring fans and air conditioning. What it doesn’t have, unlike its previous-generation counterparts is kilometres of video cabling. Aside from a tiny SDI layer for multi-view and waveform monitoring, and a video layer for emergencies, the CAR is totally IP. All hardware is connected by fibre, accessible via IP from anywhere in the Hub.
Not only has the use of IP infrastructure meant no video cabling, it has also taken months off the timeline for construction of the facility. In its entirety, the Hub took around six months to build. Segar estimates traditional video wiring would have taken six- to eight-months alone.
“I think if we built this site in SDI, we couldn’t have done it.,” he says. ““So, we needed an IP anyone-anywhere concept, a virtualised concept that says that you can sit anywhere on the network, and we wanted to have multiple sites and multiple trucks to connect over an IP network. So, we didn’t want to go back to SDI and then have latency from upconversion.
“The timeframe was very aggressive, and Fox Sports had a very aggressive timeframe. Telstra also, because at the end of the day we’ve built this in six months and so did Telstra.”
Telstra’s Distributed Production Network (DPN), connecting sports venues across the country, enters the Andrews Hub CAR via a 320Gig connection. There are also connections to NEP’s dark fibre network and Optus, allowing access to other venues outside the DPN.
When it comes to connectivity within the Hub, everything is handled through the CAR’s fibre patch.
“Anything that leaves this room to anywhere else in the building goes on our fibre patch. We don’t patch video signals around, we patch bandwidth. What ports do you need for from the core switchers and where, we do that here. The production galleries are patched here. This also extends to the two studios, and the NRL bunker.
A key component of the Andrews Hub CAR, and a platform that didn’t exist when the Hubs were being planned, is LAWO’s V matrix. Comprising software-defined virtual modules, the platform handles all IP routing, processing and multi-viewer functionality. In addition, Lawo’s Virtual Studio Manager (VSM) and ‘The WALL’ multi-viewer control provide orchestration and control as well as a common user interface for all operators.
“It’s basically a blade,” says Marc Segar. “It’s software defined, it can be anything. It’s not pre-defined, and it’s completely virtual. It’s not really a one-thing device, a virtual product can do anything.
“The great thing is when you swap a card, it learns what it was before and it comes up in the same mode. So, if you get a failure, you know longer have to carry a hundred different things.”
Also coming to the IP-enabled party was manufacturer Ross Video with its Newt family of point-of-use IP signal converters which were developed in conjunction with NEP to drive video monitoring.
The Newt converters work in a wide range of environments such as Ross DashBoard, NMOS IS-04/IS-05, and EmBER+/RAVENNA. There is also a published JSON API for custom integrations.
NOT THE END OF THE ROAD
PIC – NEP Australia’s Director of Technical Services, Marc Segar.
With the move to remote production reducing the number of personnel needing to travel around the country, does this spell the end for NEP’s fleet of outside broadcast vehicles?
“We won’t change the quantity of trucks because we’ll still have the same problem in terms of places to go to,” says Segar. “We’re not going to suddenly say out of 15 trucks we’re going to have 10; that’s just not going to work. We have upgraded seven big trucks with an IP layer. They’re basically IP trucks almost now. It’s more likely we’ll go down the path of either putting more hybrid kits together, or building more IP trucks, and then taking the old technology trucks offline. In the future, we have space for a fifth control room here, we have space for a fourth control room in Melbourne, and if we’re doing that, then we need more trucks. So, it’s more likely we’ll build more IP trucks and we’ll get the old trucks out.
“I know everyone’s looking at us at the moment sort of to see if this works, but the truth is, we’ve done 40-odd projects out of here now, and they’ve all been flawless. There’s no real boundaries. It’s about getting your head around that it’s all virtualised. You just tell me what you want, and we can do it.
“We’ve broken the myth that the EVS operator has to be sitting next to the other EVS operator. Now, we’re not doing that right now, but the end state is we’ve decentralised to the point where we just use the people where they live and we don’t fly people, and that means if there’s two people in Adelaide, two people in Melbourne, two people in Sydney, and if they’re all working as part of the same EVS team, then that’s what it’s going to be. No reason we can’t do it today, once we educate people that that’s going to work.”
SELF-SERVICE WEB SITE ADVERTISING
Manage your own ads on this web site. For more, click the button below.