Menu extends

Jun 20, 2015
Reinheitsgebot: A clear and positive influence on the definition of European media file exchange and delivery formats
By taking a look at Reinheitsgebot – the “German Beer Purity Law” – we examine how restricting a “recipe” to specific “ingredients” can result in consistent “flavours” of beer – or in this case, media. Find out why the very definition of file formats for exchange and delivery in the media industry has everything to do with the purity, or quality, of media files.

Reinheitsgebot: A clear and positive influence on

By taking a look at Reinheitsgebot – the “German Beer Purity Law” – we examine how restricting a “recipe” to specific “ingredients” can result in consistent “flavours” of beer – or in this case, media. Find out why the very definition of file formats for exchange and delivery in the media industry has everything to do with the purity, or quality, of media files.

It doesn’t take much research into either Reinheitsgebot or file specifications to realise that this title is almost complete nonsense. When Reinheitsgebot, aka the “German Beer Purity Law,” was first endorsed by the duchy of Bavaria 499 years ago (23rd April 1516) it actually had nothing to do with the purity of beer and everything to do with the price of bread – banning the use of wheat in beer to ensure that there was no competition between brewers and bakers for limited supply.

Reinheitsgebot has come to represent a mark of quality in beer and something that German brewers are very proud of, but as the law spread across what is now modern Germany in the 16th century, it actually lead to the disappearance of many highly regarded regional specialities and variations.

By contrast, the definition of file formats for exchange and delivery in the media industry has everything to do with the purity, or quality, of media files – indeed the initiative that has lead to the publication of the ARD-ZDF MXF Profiles in the German-speaking community was lead by the group looking at quality control and management.

This has represented a fairly significant change in mind-set in our approach to QC. Within reason, the file format should not really affect the “quality” of the media (assuming sufficient bit-rate). However, to have a consistent file-QC process, you need to start with consistent files, and the simplest way to do this is to restrict the “ingredients” in order to deliver a consistent “flavour” of file. By restricting the variations, we considerably simplify QC processes, mitigate risk of both QC and workflow errors occurring downstream, and reduce the cost of implementation through decreased on-boarding requirements.

This point is critical, and for illustration, one need only refer to the results of the IRT MXF plug-fest that takes place each year. At the 2014 event, outputs and interoperability of 24 products from 14 vendors, restricted to four common essence types and two wrapper types, were tested.

Even with these restrictions, a total of 4,439 tests were conducted. Assuming each test takes an average of 60 seconds, that equates to very nearly two whole man-weeks of testing before we even consider workflow-breaking issues such as time-code support, frame accuracy, audio/video off-set, etc.

Constrained media file specifications equate to far fewer variations, simplifying the on-boarding process and enabling media organizations to easily facilitate thorough automated and human QC, while focusing on the quality of the media, not the interoperability of the file.

However, the file specifications themselves may not completely answer all our problems. Referring back to the German beer market, despite the regulation being lifted in 1988 following a ruling by the European Court of Justice, many breweries and beers still claim compliance with Reinheitsgebot, even though very, very few beers actually do. We have two issues in media that are equivalent – future proofing and compliance.

When introduced, Reinheitsgebot specified three permitted ingredients – water, barley and hops. Unknowingly, however, brewers were adding another ingredient – either natural airborne yeast, or yeast cultivated from previous brews, a necessary addition for the fermentation process. Without launching into a convoluted discussion about “unknown, unknowns,” from this we learn that we have to accept the extreme difficulties of scoping future requirements.

Reinheitsgebot was replaced in 1993 by the Provisional German Beer Law, allowing for ingredients such as yeast and wheat, without which the famous Witbier (wheat beer) would not exist – one of the German beer industry’s biggest exports. Globally, this has lead to much confusion over what Reinheitsgebot compliance means, especially with many wheat beers claiming adherence. In the media industry, the UK DPP launched a compliance program run by the AMWA, but there are many more companies claiming compliance than appear on the official list.

While I suspect that many beers have been consumed in the writing of media file specifications, in reality it is unlikely that the story of the German beer purity law has had much impact – it may still have some lessons to teach us though.

And now, time for a beer! Cheers!

Note: this article also appeared in the June 2015 issue of TV Technology Europe

YOU MAY ALSO LIKE...
Dalet Solutions Earn DPP 2020 Security Certification for Production and Broadcast
Dalet, a leading provider of solutions and services for broadcasters and content professionals, continues its commitment to the highest security standards, attaining the DPP Security Marks for Production and Broadcast under ‘The DPP Committed to Security’ program. The marks certify that all Dalet products and solutions are developed, configured and deployed according to stringent DPP cyber security best practices across R&D, code safety, and operational measures. As early adopters of DPP compliance and security initiatives, both Dalet, and Ooyala - now part of Dalet - have worked with the DPP on certification and security initiatives since 2014. The company also attained its ISO/SEC 27001:2013 certification in 2018, earning the highest level of security practices across Dalet internal development processes, its product line and its practices. “Security has always been of paramount importance at Dalet. With media organizations quickly pivoting their operations to enable work from home scenarios, security has taken on an even higher level of urgency for our customers,” states Rami Pinku, Dalet Deputy General Manager, R&D Operations. “Our commitment to developing and delivering highly secure solutions that embrace industry best practices is stronger than ever. Dalet security processes start from the moment we begin developing our solutions to the time they’re delivered. We are proud to have achieved the DPP’s security marks for our solutions as these are key criteria for media organizations investing in enterprise-grade workflow solutions.” Achieving the DPP security marks demonstrates Dalet’s commitment to working towards and adhering to cyber security best practice across its entire solution range. Powering exceptional user workflows from enterprise productions to OTT preparation and finished asset distribution, Dalet’s line of DPP accredited media supply chain and broadcast solutions, including Dalet Galaxy five, the Ooyala Flex Media Platform and Dalet’s latest SaaS offerings, Dalet StoreFront, Dalet Media Cortex and Dalet Galaxy xCloud, offer secure, hybrid and highly scalable workflows on-premises and in the cloud. "We're delighted that Dalet has been awarded the DPP's Committed to Security mark for both Broadcast and Production," says Rowan de Pomerai, DPP Head of Delivery & Growth. "Building on the great work they did with the Ooyala Flex Media Platform, all Dalet products are now developed in line with our Security guidelines, meaning that they remain part of a community of forward-thinking companies demonstrating a clear commitment to cybersecurity best practice, and to playing their part in building a more secure media supply chain." About the DPP Committed to Security program The DPP launched the Committed to Security program in October 2017 to help technology providers advance, hone and demonstrate their commitment to security best-practices. Participants are assessed according to a rigorous set of controls specifically applied within the categories of Production, including policies and procedures, physical security, incident planning, recovery management, IT security, business continuity and other areas; and Broadcast, including documentation & testing, authentication and controls. For more information go to https://www.dalet.com/platform and https://www.thedpp.com/security. About Dalet Digital Media Systems Dalet solutions and services enable media organizations to create, manage and distribute content faster and more efficiently, fully maximizing the value of assets. Based on an agile foundation, Dalet offers rich collaborative tools empowering end-to-end workflows for news, sports, program preparation, post-production, archives and enterprise content management, radio, education, governments and institutions. Dalet platforms are scalable and modular. They offer targeted applications with key capabilities to address critical functions of small to large media operations - such as planning, workflow orchestration, ingest, cataloging, editing, chat & notifications, transcoding, play out automation, multi-platform distribution and analytics. The integration of the Ooyala Flex Media Platform business has opened vast opportunities for Dalet customers to deploy successful strategies that better address their audiences with agile multi-platform content distribution in a wider range of markets, such as sports for teams and leagues, brands and corporate organizations, as well as Media and Entertainment companies looking to scale up their digital offerings. Dalet solutions and services are used around the world at hundreds of content producers and distributors, including public broadcasters (BBC, CBC, France TV, RAI, TV2 Denmark, RFI, Russia Today, RT Malaysia, SBS Australia, VOA), commercial networks and operators (Canal+, FOX, MBC Dubai, Mediacorp, Fox Sports Australia, Turner Asia, Mediaset, Orange, Charter Spectrum, Warner Bros, Sirius XM Radio), sporting organizations (National Rugby League, FIVB, Bundesliga) and government organizations (UK Parliament, NATO, United Nations, Veterans Affairs, NASA). Dalet is traded on the NYSE-EURONEXT stock exchange (Eurolist C): ISIN: FR0011026749, Bloomberg DLT:FP, Reuters: DALE.PA. Dalet® is a registered trademark of Dalet Digital Media Systems. All other products and trademarks mentioned herein belong to their respective owners.
Life before and after DPP (Digital Production Partnership)
People that know me will be aware that file-based workflows are a passion of mine. Ten years ago I was co-author of the MXF (Media Exchange Format) specification and ever since I have been engaged in taking this neatSMPTE standard and using it to create a business platform for media enterprises of every size and scale. This is why I’m so excited by the Digital Production Partnership (DPP): it represents the first ratified national Application Specification of the MXF standard and is set to revolutionize the way that media facilities and broadcasters work.To explain what I mean, let’s compare life with a DPP ecosystem to life without. Less pain to feel the gain 
In a standardized DPP world, there would be a limited amount of pain and cost felt by everybody but this would be shared equally amongst the organizations involved and it would be a limited cost, which is incurred only once. After this point, our industry has a fantastic common interchange format to help encourage partnerships and build businesses. In an unstandardized world, where different facilities have decided to use different tools and variants of MXF or other formats, the major cost becomes the lack of third-party interoperability. Each time content is exchanged between different facilities, a media transcode or rewrap in that format is required. This means that all vendors in all the facilities will ultimately support all the file formats andmetadata. The engineering required to implement and test takes time and costs money on an on-going basis. Interoperable metadata helps the content creator 
In a world that has adopted DPP, media and metadata interoperability is not an issue since the format is built on a strong, detailed common interchange specification. In this homogeneous scenario the resources that would have been used in the interoperability engineering process can be used in more creative and productive ways, such as programme making. Programme making is a process where most broadcasters utilise external resources. In a world without DPP, whenever a broadcaster or production facility receives a new file from an external facility, such as a Post House, the question must be asked whether this file meets the requirements of their in-house standard. That evaluation process can lead to extra QC costs in addition to possible media ingest, transcoding, conformance and metadata re-keying costs that need to be taken into account. Building a business platform
 This heterogeneous environment is an issue not just for interaction with external facilities: often different departments within the same major broadcaster will adopt slightly different file standards and metadata making interoperability a big issue to them. As a result, today only about 70 per cent of transactions within companies are file-based – the remainder employ tape. However, this is much higher than where external agencies are involved – here, only 10 – 15 per cent of transactions are file-based. The essence of the problem is the lack of a common interchange format to enable these transactions. DPP is the first open public interchange format that is specifically designed to address this issue. DPP is intended to transform today’s 20 per cent trickle into an 80 per cent flood in the shortest time. To find out more about DPP and how it can transform the way your operation works and also your effectiveness working with other organizations read AmberFin’s White Paper on DPP.
5 reasons why media delivery standards might be good for your business
Like me, I am sure that you have been to a restaurant in a group and everyone orders from the set menu EXCEPT for that one person who orders the exotic, freshly prepared fugu, which requires an extra 30 minutes of preparation from a licensed fugu chef so that the customers don't die eating it. Restaurant etiquette means that our main course is served at the same time, forcing everyone to spend a long time hungry, waiting for the special case. And if you split the bill equally, the special case becomes subsidised by the people wanting the set meal. Does this model relate to the media industry? Is there a cost for being special? How can we reduce that cost? What gets done with the cost savings? How can you help? Fortunately those 5 questions lead into 5 reasons why delivery standards might be a good idea. 1. The set meal is more efficient than the a la carte I must confess that when I write this blog while hungry there will be a lot of food analogies. I'm quite simple really. In the "set meal" case - you can see how it's easier for the kitchen to make a large volume of the most common meal and to deliver it more quickly and accurately than a large number of individual cases. In the file delivery world, the same is true. By restricting the number of choices to a common subset that meet a general business need, it is a lot easier to test the implementations by multiple vendors and to ensure that interoperability is maximised for minimum cost. In a world where every customer can choose a different mix of codecs, audio layout, subtitle & caption formats, you quickly end up with an untestable mess. In that chaotic world, you will also get a lot of rejects. It always surprises me, how few companies have any way of measuring the cost of those rejects, even though they are known to cause pain in the workflow. A standardised, business-oriented delivery specification should help to reduce all of these problems. 2. Is there a cost for being special? I often hear the statement – "It's only an internal format - we don't need to use a standard". The justification is often that the company can react more quickly and cheaply. Unfortunately, every decision has a lifespan. These short-term special decisions often start with a single vendor implementing the special internal format. Time passes and then a second vendor implements it, then a third. Ultimately the custom cost engineering the special internal format is spent 3 or 4 times with different vendors. Finally the original equipment will end of life and the whole archive will have to be migrated. This is often the most costly part of the life cycle as the obsolete special internal format is carefully converted into something new and hopefully more interchangeable. Is there a cost of being special? Oh yes, and it is often over and over again. 3. How can we reduce costs? The usual way to reduce costs is to increase automation and to increase "lights out" operation. In the file delivery world, this means automation of transcode AND metadata handling AND QC AND workflow. At Dalet and AmberFin, all these skills are well understood and mastered. The cost savings come about when the number of variables in the system is reduced and the reliability increases. Limiting the choices on metadata, QC metrics, transcode options, workflow branches increases the likelihood of success. Learning from experiences of the Digital Production Partnership in the UK, it seems that tailoring a specific set of QC tests to a standardised delivery specification with standardised metadata will increase efficiency and reduce costs. The Joint Task Force on File Formats and Media Interoperability is building on the UK's experience to create an American standard that will continue to deliver these savings 4. What gets done with the cost savings? The nice thing about the open standards approach is the savings are shared between the vendors who make the software (they don't have to spend as much money testing special formats) and the owners of that software (who spend less time and effort on-boarding, interoperability testing and regression testing when they upgrade software versions.) 5. How can you help? The easiest way is to add your user requirements to the Joint Task Force on File Formats and Media Interoperability list. These user requirements will be used to prioritise the standardisation work and help deliver a technical solution to a commercial problem. For an overview of some of the thinking behind the technology, you could check out my NAB2014 video on the subject, or the presentation given by Clyde Smith of Fox. Until next time.
How to bring standards to your organisation
Back in the 1990s, I was told of an old maxim: "If you can't win the market place, win the standard." I thought that this was a cynical approach to standardisation until we looked through some examples of different markets where there are a small number of dominant players (e.g., CPUs for desktop PCs, GPU cards, tablet / smartphone OS) versus markets where there is enforced cooperation (Wi-Fi devices, network cabling, telephone equipment, USB connectivity). So, how does this affect technology in the media industry, and how can you use the power of standards in your organisation? It seems that the media technology industry hasn't made its mind up about what's best. We have come from a history that is strong in standardisation (SDI, colour spaces, sampling grids, etc.), and this has created a TV and film environment where the interchange of live or streaming content works quite well, although maybe not as cheaply and cleanly as we would like. When the material is offline or file-based, there are many more options. Some of them are single-vendor dominant (like QuickTime), some are standards-led (like MXF), some are open source (Ogg, Theora) and others are proprietary (LXF, FLV). Over any long timeframe, commercial strength beats technical strength. This guiding principal should help explain the dynamics of some of the choices made by organisations. Over the last 10 years, we have seen QuickTime chosen as an interchange format where short-term "I want it working and I want it now" decisions have been dominant. In other scenarios – as in the case of "I am generating thousands of assets a month and I want to still use them in six years time when Apple decides that wearables are more important than tablets" – MXF is often the standard of choice. Looking into the future, we can see that there are a number of disruptive technologies that could impact decision-making and dramatically change the economics of the media supply chain: IP transport (instead of SDI) High Dynamic Range (HDR) video 4k (or higher) resolution video Wide colour space video HEVC encoding for distribution High / mixed frame rate production Time Labelling as a replacement for timecode Specifications for managing workflows Some of these are clearly cooperative markets where long-term commercial reality will be a major force in the final outcome (e.g., IP transport). Other technologies could go either way – you could imagine a dominant camera manufacturer “winning” the high / mixed frame rate production world with a sexy new sensor. Actually, I don't think this will happen because we are up against the laws of physics, but you never know – there are lots of clever people out there! This leads us to the question of how you might get your organisation ahead of the game in these or other new technology areas. In some ways being active in a new standard is quite simple – you just need to take part. This can be costly unless you focus on the right technology and standards body for your organisation. You can participate directly or hire a consultant to do this speciality work for you. Listening, learning and getting the inside track on new technology is simply a matter of turning up and taking notes. Guiding the standards and exerting influence requires a contributor who is skilled in the technology as well as the arts of politics and process. For this reason, there are a number of consultants who specialise in this tricky but commercially important area of our business. Once you know “who” will participate, you also need to know “where” and “how.” Different standards organisations have different specialties. The ITU will work on the underlying definition of colour primaries for Ultra High Definition, SMPTE will define how those media files are carried and transported, and MPEG will define how they are used during encoding for final delivery. Figuring our which standards body is best suited for the economic interests of your organisation requires a clear understanding of you organisation’s economics and some vision about how exerting influence will improve those economics. Although a fun topic, it's a little outside today's scope! So how do you bring standards to your organisation? Step 1: join in and listen Step 2: determine whether or not exerting influence is to your advantage Step 3: actively contribute Step 4: sit back and enjoy the fruits of your labour For more on the topic, don't forget to listen to our webinars! Coming soon, I'll be talking about Business Process Management and standards – and why they matter. Until the next one...
BEST…NAB…EVER!!!!!
The other day, a member of our talented development team commented, quite accurately, that every time we return from an NAB Show, we nearly always refer to it as the biggest, busiest and best NAB ever. If you’ve ever watched or read one of my presentations or blogs on workflow, you may recollect that I’m a fan of the Toyota Production System and the “Kaizen” concept of continuous improvement. However, I do confess that, following my colleagues’ observation, I momentarily felt a certain amount of pressure to come back from NAB 2015 with evidence that it really was bigger, busier and better than previous years. However, earlier today I was talking to the editor of one of our excellent industry magazines about the most likely themes and trends for this year’s show and something struck me. Although I’m not much of a fan of “buzzword bingo,” given the host of announcements we at Dalet have for this year’s show, I’d place a bet on us sweeping the board. Even before the show, we’ll bring UHD to Dalet AmberFin – supporting UHD inputs in our next release at the end of March. By decoupling format from transport mechanism, Video over IP is one of the most revolutionary changes to the industry in some time, and our Dalet Brio video server platform is spearheading that charge. Building on all of this, Dalet Galaxy, our media asset management platform, continues to facilitate and enhance collaborative workflows with new features for user interaction and geographically dispersed operations –I can barely contain myself from mentioning the “C” word! It doesn’t stop there though. Back in September, we got quite emotional about being one of the first vendors to have a product certified for the creation of UK DPP files. The DPP has led the way in specifying standards and operational guidelines for file delivery and as other regions has followed, Dalet has been right there supporting them. Demonstrating our continued commitment to international standards that improve, ease and simplify the lives of our customers, we’ve now implemented the FIMS capture service in the Brio video server. I believe that initiatives like FIMS become ever more important as the video world increasingly leverages IT technology and, specifically, interaction between control and capture devices as we move to an era of hybrid SDI and IP acquisition. Despite regulatory rulings in the US and elsewhere, captioning and subtitling technology has seen little innovation in the last few years. Since Dalet and AmberFin came together a year ago, we’ve really focused on this as an area where our knowledge and expertise can benefit the industry as a whole. We’re now ready to show you what we’ve been up to and how we can simplify captioning workflows and bring them into multi-platform, multi-version workflows in an effective and efficient way. You’re probably aware of the Dalet Academy, which was launched with much fanfare in January this year. The response from the wider industry has simply been immense, and we now have many thousands of followers subscribed to the Bruce’s Shorts videos and reading our educational blog. For NAB 2015, we’ll be donning our robes and mortarboards to bring the Dalet Academy to the stage, live on our booth (SL4525). Bruce will be there – in his actual shorts – to present special live editions of the video series with support from other Dalet and industry experts for more short seminars. All of the presentations at the show will be followed by a special round-table discussion (limited seating). And while you’re keeping your media knowledge in good shape, there will also be an opportunity to win prizes that are sure to keep you in good shape too! To make sure the excitement doesn’t overwhelm too much, we’re keeping a couple of bits of news to ourselves until the show itself, but if you want to find out more on any of the topics I’ve touched on here, be sure to get in touch, book an appointment, or read more on our dedicated NAB page. As for our development team – sorry guys, I can already tell you that this year is going to be the biggest, busiest and best NAB Show so far!
Could your MXF files be infected with a virus?
We all know not to click on those shifty-looking attachments in emails, or to download files from dubious websites, but as file delivery of media increases, should we be worried about viruses in media files? In the case of the common computer virus, the answer is “probably not” – the structure of media files and applications used to parse or open MXF, QuickTime and other files do not make “good” hosts for this type of virus. Compared to an executable or any kind of XML-based file, media files are very specific in their structure and purpose – only containing metadata, video and audio – with any element labeled appropriately sent to the applicable decoder. Any labels that are not understood or supported by the parser are simply ignored. However, this behavior of ignoring unsupported or unrecognized labels facilitates the existence of “dark metadata,” and this is a potential area of weakness in the broadcast chain. Dark metadata isn’t necessarily as menacing as the name could suggest and is most commonly used by media equipment and software vendors to store proprietary metadata that can be used downstream to inform dynamic processes – for example, to change the aspect ratio conversion mode during up or down conversion, or audio routing in a playout video server. When you know what dark metadata you have, where it is and what it means, it can add value to the workflow chain. Since dark metadata will usually be ignored by parsers that don’t understand/support the proprietary data it carries, it can also be passed through the media lifecycle in a completely harmless way. However, if you are not aware of the existence of dark metadata and/or the values of the data it carries, then there is a risk that processes in the media path could be modified or activated unintentionally and unexpectedly. In this case, the media is in some way carrying a virus and in the worst case, could result in lost revenue. The anti-virus software installed on your home or work PC isn’t going to be much help in this instance, but there are simple steps that can be taken to ensure that you don’t fall foul of “unknown unknowns.” Implement a “normalization” stage at the entry point for media into your workflow. You can read other articles in this blog about the benefits of using a mezzanine file format, but even if files are delivered in the same format you use in-house, a simple re-wrapping process to “clean” and normalize the files can be a very lightweight process that adds little or no latency into the workflow. Talk to your suppliers and vendors to make sure you’re aware of any proprietary metadata that may be being passed into your workflow. If you have an automated file-QC tool, check whether it has a “dark metadata” test and switch it on – unless you definitely use proprietary metadata in your workflow, this won’t generate false positives and shouldn’t add any significant length to the test plan. We’ll be looking at some of the other security concerns in future blogs, but as long as you know your dark metadata, there’s little risk of viral infection from media files.
HPA: Mapping the Future, One Pixel at a Time
I love the HPA Tech Retreat. It is the most thought provoking conference of the year, one where you're guaranteed to learn something new, meet interesting people and get a preview of the ideas that will shape the future of the industry. Here are the six most interesting things I learned this year. Collaborating competitors can affect opinions At this year’s HPA Tech Retreat, I had the honour of presenting a paper with John Pallett from Telestream. Despite the fact that our products compete in the market place, we felt it important to collaborate and educate the world on the subject of fractional frame rates. 30 minutes of deep math on drop frame timecode would have been a little dry, so we took some lessons from great comedy double acts and kept the audience laughing, while at the same time pointing out the hidden costs and pitfalls of fractional frame rates that most people miss. We also showed that there is a commercial inertia in the industry, which means the frame rate 29.97i will be with us for a very long time. In addition to formal presentations, HPA also features breakfast round tables, where each table discusses a single topic. I hosted two great round tables, with John as a guest host on one, where the ground swell of opinion seems to be that enforcing integer frame rate above 59.94fps is practical, and any resulting technical issues can be solved – as long as they are known. I will never be smart enough to design a lens Larry Thorpe of Canon gave an outstanding presentation of the design process for their latest zoom lens. The requirements at first seemed impossible: design a 4K lens with long zoom range that is light, physically compact, and free from aberrations to meet the high demands of 4K production. He showed pictures of lens groupings and then explained why they couldn't be used because of the size and weight constraints. He went on to show light ray plots and the long list of lens defects that they were battling against. By the end of the process, most members of the audience were staring with awe at the finished lens, because the design process seemed to be magical. I think that I will stick to the relative simplicity of improving the world's file-based interoperability. Solar flares affect your productions We've all seen camera footage with stuck or lit pixels and, like most people, we probably assumed that they were a result of manufacturing defects or physical damage. Joel Ordesky of Court Five Productions presented a fascinating paper on the effects of gamma photons, which, when passing through a camera’s sensor, cause the sensor to permanently impair individual pixels. This is something that cannot be protected against unless you do all of your shooting underground in a lead lined bunker. Joel presented some interesting correlations between sun spot activity and lit pixels appearing in his hire stock, and then showed how careful black balance procedures can then reduce the visibility of the issue. UHD is coming – honest The HPA Tech Retreat saw a huge range of papers on Ultra High Definition (UHD) issues and their impacts. These ranged from sensors to color representation to display processing, compression, high frame rates and a slew of other issues. I think that everyone in the audience recognised the inevitability of UHD and that the initial offering will be UHDTV featuring resolution improvements. This is largely driven by the fact that UHD screens seem to be profitable for manufacturers; soon enough they will be the only options available at your local tech store (that’s just good business!). The displays are arriving before the rest of the ecosystem is ready (a bit like HDTV), but it also seems that most of the audience feels better colour and high dynamic range (HDR) is a more compelling offering than more pixels. For me, the best demonstration of this was the laser projector showing scenes in true BT2020 wide colour range. First we saw the well-known HDTV Rec.709 colour range and everything looked normal. Next up was the same scene in BT2020 – and it was stunning. Back to Rec.709, and the scene that looked just fine only seconds before now appeared washed out and unsatisfactory. I think HDR and rich colors will be addictive. Once you've seen well-shot, full color scenes, you won't want to go back to Rec.709. The future is looking very colourful. Women are making more of an impact in the industry (Hooray!) There were three all-women panels at this year's HPA, none of which were on the subject of women in the industry. This was a stark contrast to the view of women in the industry as shown on a 1930s documentary of the SMPTE Conference, where men with cigars dominated the proceedings and women were reduced to participating in the chattering social scene. This contrast was beautifully and ironically highlighted by Barbara Lange (Executive Director of SMPTE) and Wendy Aylesworth (President of SMPTE 2005-2015), who hosted their panel in bathrobes with martini glasses, while explaining the achievements of the society over the year. If you haven't yet contributed to the SMPTE documentary film project or the SMPTE centennial fund, it's time to do so now. These funds will help support the next, diverse generation of stars. IMF and DPP are a symbiotic pair One of the most interesting panels was on the Interoperable Mastering Format (IMF) and the Digital Production Partnership (DPP) interchange format (and yes, this was in fact one of my panels!). One format’s purpose is to distribute a bundle of files representing several versions of one title. The other is designed to create a finished, single file with ingest-ready metadata, where the file can be moved to playout with virtually no changes. Both formats have a strong foothold in the life cycle of any title and are likely to form the strongest symbiotic relationship as we move into the future. One thing that I pointed out to the audience is that the DPP has done a huge amount of work educating UK production and postproduction houses about the change management that is required for file-based delivery. They have written a wonderful FREE guide that you can download from their website. All in all, the HPA Tech Retreat is a wonderful event with so much information flowing that it takes weeks to absorb it all. I must confess though, that one of the highlights for me was being able to cycle up the mountain every morning before breakfast. It meant that I could go back for seconds of all the wonderful cake that was on offer. Happy days! Until next time – don't forget about our UHD webinar, happening today. If you didn’t sign up in time, drop us a line at academy@dalet.com and ask for a re-run. The more people that ask, the more likely that we'll do it!
Does it take you 1/2 million years to test your workflow?
It is now obligatory to start every broadcast technology blog post, article or presentation with a statement reminding us that we are now living in a multi-format, multi-platform world, where consumers want to view the content they choose, when they want it, where they want it, on the device they want. However, unlike other marketing platitudes, this one is actually true: Many of us in this industry spend our days trying to develop infrastructures that will allow us to deliver content to different platforms, ageing prematurely in the process because to be honest, it's a really hard thing to do. So why is it so hard? Why is it so hard? Let me explain: For each device, you have to define the resolution: a new iPad has more pixels than HDTV, for example (2048 wide), and is 4:3 aspect ratio. Android phones have different screen sizes and resolutions. Don’t even get me started on interlaced or progressive. That video has to be encoded using the appropriate codec – and of course different devices use different codecs. Along with the pictures there will be sound. Which could be in mono, stereo or surround sound, which in turn could be 5.1, 7.1 or something more exotic. The sound could be encoded in a number of different ways. Digital audio sampling could be at 44.1kHz or 48kHz and a whole range of bit depths. Then the audio and video need to be brought together with the appropriate metadata in a wrapper. The wrapper needs to be put into a delivery stream. If it is for mobile use, we now routinely adopt one of the three different adaptive bitrate formats, which means essentially we have to encode the content at three different data rates for the target device to switch between. If you want to achieve the admirable aim of making your content available on all common platforms, then you have to take into consideration every combination of resolution, video codec, audio codec, track layout, timecode options, metadata and ancillary data formats and bitrate options. This is a very large number. And it does not stop there. That is only the output side. What about the input? How many input formats do you have to support? Are you getting SD and HD originals? What about 2k and, in the not too distant future, 4K originated material? If you are producing in-house, you may have ARRI raw and REDCODE (R3D) files floating around. The content will arrive in different forms, on different platforms, with different codecs and in different wrappers. We are on to the third revision of the basic MXF specification, for example. Any given end-to-end workflows could involve many, many thousands of input to output processes, each with their own special variants of audio, video, control and metadata formats, wrappers and bitrates. Each time a new input or output type is defined the number increases many-fold. Quality Control All of which is just mind-boggling. Until you consider quality control. If you were to test, in real time, every variant of, say, a three minute pop video, it would take a couple of hundred years. This is clearly not going to happen. It’s all right, I hear you say. All we need do is define a test matrix so that we know we can transform content from any source to any destination. If the test matrix works, then we know that real content will work, too. Well, up to a point. I have done the calculations on this and, to complete a test matrix that really does cover every conceivable input format, through every server option, to every delivery format for every service provider, on every variant of essence and metadata, it is likely to take you half a million years. Maybe a bit more. So are you going to start at workflow path one and test every case, working until some time after the sun explodes? Of course not. But what is the solution? Do you just ignore all the possible content flows and focus on the relatively few that make you money? Do you accept standardized processing which may make you look just like your competitors; or do you implement something special for key workflows even though the cost of doing it – and testing it – may be significant? We have never had to face these questions before. Apart from one pass through a standards converter for content to cross the Atlantic, everything worked pretty much the same way. Now we have to consider tough questions about guaranteeing the quality of experience, and make difficult commercial judgments on the right way to go. If you want to find out more about how to solve your interoperability dilemma, why don't you register for our next webinar on: Wednesday 29th January at: 1pm GMT, 2pm CET, 8am EST, 5am PST OR 5pm GMT, 6pm CET, 12pm EST, 9am PST I hope you found this blog post interesting and helpful. If so, why not sign-up to receive notifications of new blog posts as they are published?
5 Reasons why we need more than ultra HD to save TV
If you were lucky (or unlucky) enough to get to CES in Las Vegas this year, then you will know that UHD (Ultra High Definition TV) was the talking point of the show. By and large the staff on the booths were there to sell UHD TVs as pieces of furniture and few of them know the techno-commercial difficulties of putting great pictures onto those big, bright, curved(?) and really, really thin displays: In my upcoming webinar on the 29th January I will be looking into the future and predicting some of the topics that I think will need to be addressed over the next few years if TV as we know it is to survive. 1. Interoperability The number of screens and display devices is increasing. The amount of content available for viewing is going up but the number of viewers is not changing greatly. This means that we either have to extract more revenue from each user or reduce the cost of making that content. Having systems that don’t effectively inter-operate adds cost, wastes time and delivers no value so the consumer. Essence interoperability (video & audio) is gradually improving thanks to education campaigns (from AmberFin and others) as well as vendors with proprietary formats reverting to open standards because the cost of maintaining the proprietary formats is too great. Metadata interoperability is the next BIG THING. Tune in to the webinar to discover the truth about essence interoperability and then imagine how much unnecessary cost exists in the broken metadata flows that exists between companies and between departments. 2. Interlace must die UHD may be the next big thing, but just like HDTV it is going to have to show a lot of old content to be a success. Flick through the channels tonight and ask yourself “How much of the content was shot & displayed progressively”. On a conventional TV channel the answer is probably “none”. Showing progressive content on a progressive screen via an interlaced TV value chain is nuts. It reduces quality and increases bitrate. Anyone looking at some of the poor pictures shown at CES will recognise the signs of demonstrations conceived by marketers who did not understand the effects of interlace on an end to end chain. Re-using old content involves up-scaling & deinterlacing existing content – 90% of which is interlaced. In the webinar, I’ll use AmberFin’s experience in making the world’s finest progressive pictures to explain why interlace is evil and what you can do about it. 3. Automating infrastructure Reducing costs means spending money on the things that are important and balancing expenditure between what is important today and what is important tomorrow. There is no point in investing money in MAMs and Automation if your infrastructure won’t support it and give you the flexibility you need. You’ll end up redesigning your automation strategy forever. The folks behind xkcd.com explain this much more succinctly and cleverly than I could ever do. In the webinar, I’ll explain the difference between different virtualization techniques and why they’re important. 4. Trust confidence & QC More and more automation brings efficiency, cost savings and scale, but also means that a lot of the visibility of content is lost. Test and measurement give you the metrics to know about that content. Quality Control gives you decisions that can be used to change your Quality Assurance processes. These processes in turn allow your business to deliver media product that delivers the right technical quality for the creative quality your business is based on. So here’s the crunch. The more you automate, the less you interact with the media, the more you have to trust the metadata and pre-existing knowledge about the media. How do you know it’s right? How do you know that the trust you have in that media is founded? For example. A stranger walks up to you in the street and offers you a glass of water. Would you drink it? Probably not. If that person was your favourite TV star with a camera crew filming you – would you drink it now? Probably? Trust means a lot in life and in business. I’ll explore more of this in the webinar. 5. Separating the pipe from the content If, like me, you’re seeing more grey hair appearing on the barber’s floor with each visit then you may remember the good old days when the capture standard (PAL) was the same as the contribution standard (PAL) and the mixing desk standard (PAL) and the editing standard (PAL) and the playout standard (PAL) and the transmission standard (PAL). Today we could have capture format (RED), a contribution standard (Aspera FASP), a mixing desk standard (HDSDI), an editing standard (MXF DNxHD),a playout standard (XDCAM-HDSDI) and a transmission standard (DVB-T2) that are all different. The world is moving to IP. What does that mean? How does it behave? A quick primer on the basics will be included in the webinar. Why not sign up below before it’s too late? Places are limited – I know it will be a good one. Register for our next webinar on: Wednesday 29th January at: 1pm GMT, 2pm CET, 8am EST, 5am PST OR 5pm GMT, 6pm CET, 12pm EST, 9am PST ‘til next time. I hope you found this blog post interesting and helpful. If so, why not sign-up to receive notifications of new blog posts as they are published?
Digital Production Partnership (DPP) - A Broadcaster's Perspective
Recently, we staged a webinar at AmberFin in partnership with ATG Broadcast which focussed on theDigital Production Partnership (DPP) file standards. We had a number of contributors giving the perspectives of a service provider, a media facility and a broadcaster. The broadcaster’s perspective was provided by Shane Tuckerfrom UK broadcaster, Channel 4. The broadcaster does not produce content itself, however it does commission a great deal of content. Channel 4 is not alone in the UK commissioning market – together with the BBC and ITV, Shane explained how it identified the need for a joined up approach. DPP - Strength through a unified approach
 The UK broadcasters that established DPP have a desire to establish the means for shared learning and best practice with other UK broadcasters. Also, if UK broadcasters are unified in their adoption of digital file-based workflows, they can exert a greater influence outside of the UK. Both of these opinions were confirmed by Shane Tucker in the webinar. Another key advantage for our industry focuses on production companies. Major production companies supply several UK broadcasters and it will be a big benefit to them if we can standardise on one media interchange standard. What benefits does DPP bring to Channel 4?
 To start with DPP just makes sense. This view was confirmed by Shane Tucker, who said in the Webinar that DPP represents a common file format based on established standards (MXF, SMPTE, EBU) and has been established in conjunction with major UK broadcasters, screen & production companies. Furthermore, Shane highlighted that DPP offers a common descriptive metadata schema. It offers the ability to access, process and automate metadata within digital file-based workflows is so important. It creates improved efficiency associated with automated workflows between broadcasters and their trusted suppliers. It cuts down on the need for data re-entry and speeds up material transfer and processing from delivery to playout/CDN avoiding unnecessary transcoding. Automated QC workflows 
Another advantage of DPP is the potential to capitalise on automatic QC workflows with the production company or facility. In the webinar, Shane Tucker pointed out that there is a strong likelihood that QC processes will have been performed at numerous stages in the workflow before the media file reaches Channel 4 so any further QC cycles are unnecessary. Shane concluded his contribution to the webinar by highlighting a number of challenges that remain in the successful adoption of DPP, not least the support needed from equipment vendors. At AmberFin, we recognise this and we’re straining every sinew in our efforts to support this fantastic UK initiative. To see the webinar in full, please click "Watch the Webinar" button below.
How to bring your archive back to life and create additional revenue
As part of our continuing conversations regarding file-based workflows and the various challenges they bring with them, today's blog by Dalet's VP of Marketing Raoul Cospen looks at one of the main challenges we all have today: archive. How do you utilize your current archive? Is it solely for preservation, or legal compliance, perhaps? Do you sometimes feel that you are sitting on valuable content in your archives that could be repurposed to give you an additional revenue stream, if only it didn’t require a team of archeologists to dig it up? There’s nothing more frustrating than not finding content you’re sure is in your archives, if only you knew where exactly. Whether you’re looking for completed shows to round out your online offerings or b-roll for a new show in production, sometimes it almost feels easier and cheaper to go out and shoot again rather than brave a disorganized archival system. Metadata is the key So, what’s the secret to maximize the value of your archive? Metadata, of course! However, today’s metadata are vastly more complex than in the early days of tapeless workflows when a single metadata set equated to one media file. With the expansion of file-based media, multiple files are required to create different formats (e.g., distribution versions) or resolutions (e.g., proxy, high resolution) of the same content. To that mix, we must also add subtitles, language tracks and captions for various outputs. What’s increasingly clear is that individual files make little sense in isolation, i.e., a German language subtitle for a premium English language movie holds little value without relevance to its constituent parts. Of course metadata alone is not enough: you’ve got to have a MAM system that can intelligently interpret it, and then optimize it so that the media most valuable to you is the most easily accessible. Create a metadata model It all starts with mapping the unique needs of your organization by creating a metadata model. This may sound difficult, but it doesn’t need to be. First, you need to agree what metadata is crucial to all of your assets, and then put rules in place that ensure that metadata is entered, either manually (by not allowing loggers/archivists to save assets without certain fields filled out) or automatically (computer generated). Don’t forget about relationships too; does this content relate to similar content in your facility? (e.g., part of a certain season, series or global event). When searching for content in the future, you want a system that can define crucial relationships between content as well as utilize basic search terms. Create your data model Creating a data model is equally important. You need to know where your content is going to be used and in which format. For instance: a sports game might be the actual live game as it was broadcasted live, plus the highlights that go to tablets, plus the five-minute web promo, plus the post-produced game dubbed in a different language for a foreign TV rebroadcast, etc. So you need to know what your different distribution packages are going to be and make sure that you configure your workflow so it is able to keep all those elements under the same assets and have the business rules that will automatically distribute the right versions to the right targets. The right MAM A MAM system should in effect “know your facility” through rules you supply that state facts such as how fast access needs to be, retrieval times for various storage subsystems (online, near-line, tape), and the resulting business cost for each storage tier. Obviously, the system must have superior integration with third-party systems and tools as well. Regardless of media type, whether for news, sports, radio, long form programs, etc., it is imperative to think beyond simply storing files on tape libraries and disks. Instead, we need to think in terms of access and availability so that your archived content can continue to add value today and in the future. So you see, you don't need any kind of dark magic to re-animate your archives after all. You have to think about what you want to achieve with your archives and how you will further extract value from them. With the addition of rich, consistently applied metadata and the right system to manage it, your archives can come back to life as a source of valuable content and become an integral part of your workflow and continued revenue. Until next time! Raoul
Why is Hollywood interested in DPP?
The Digital Production Partnership (DPP) is a thoroughly British concept. Its major sponsors include the BBC, ITV, Channel 4 and Sky. From its earliest inception, DPP was an initiative forged by the British broadcast industry, for the British broadcast industry. So why is it that in my day-to-day dialogue with executives in and around the Hollywood studio community DPP is frequently raised and I’m asked for regular updates on its progression? Without doubt, AmberFin has been a big supporter of DPP initiative from day one. With people on our team such as my colleague Bruce Devlin - who is co-author of SMPTE’s MXF specification - we really know our onions when it comes to intra- and inter-company file-based media workflows, so who better to ask than AmberFin? But I believe this US interest in an inherently UK initiative goes much deeper than technical curiosity and is fuelled by commercial considerations on a global scale. Can the DPP initiative support a global industry? As we all know, today’s broadcast media market is entirely global. International markets offer tremendous opportunities for broadcasters to maximise their revenues. The BBC’s Top Gear programme, for example, has become a global phenomenon and is watched by more than 350 million viewers in 170 countries – each broadcaster in each market having their own delivery specification and without doubt, there is a significant amount of duplicated pain associated with the necessary testing of delivery specs – especially when upgrading software and systems. In order to maximise global revenues, more and more versions of material are created to satisfy this demand. Standards such as SMPTE’s IMF (Interoperable Master Format) address the need to “publish” multiple versions of a media asset for shipping to the ‘despatch’ company, which is typically a Post House. Now standards such as DPP in the UK and the tongue-tripping, US-originated Joint Task Force on File Formats and Media Interoperability (JTFFFMI) provide delivery specifications for a commercially significant territory. The underlying ethos of both of these initiatives is that they reduce overall system complexity without affecting business flexibility and through this they make very sound commercial sense for anybody involved in this market, on either side of the Mill Pond. JTFFFMI – now there’s an acronym to be proud of! The JTFFFMI is a new industry group that has recently been formed by AMWA, SMPTE, NABA, 4A's, ANA, EBU and the IABM. The kick-off meeting took place last week in New York and I attended. So far, it is too early to tell if any of the practices and specifications deployed by the DPP will be utilised in the US, but without doubt, the DPP is providing a valuable testing ground and insight as to the best practices and potential pitfalls that manifest themselves in this area. Having been involved with the DPP from the earliest stages, I applaud their approach. The DPP’s intention was never to be dictatorial and impose its preferred flavour of MXF. It has created a forum where all interested parties, including broadcasters, facilities and equipment vendors, can come together and evolve a set of application specifications that are most appropriate for the UK. What will happen on 15 October 2014? Without doubt, not everything that is appropriate to the UK will necessarily be equally appropriate to other regional markets, but what the DPP has achieved is to break the back of this herculean challenge and create a blueprint that will help any other territories develop their own application specifications for MXF-based file-based workflows. This time, it really is rocket science and this is why Hollywood, and indeed, many other areas of North America are closely watching the development and deployment of DPP. DPP Deadline Day – 15th October 2014 – is fast approaching. The world is watching to see what will happen – will they be watching your organization? If you would like more information about the DPP, then a very good starting point is AmberFin’s Technical White Paper, which you can download for free here -http://www.amberfin.com/academy/white-papers.