We've retooled the 3D Building Pipeline

This is a long post, so grab your favorite beverage and settle in.

Over the last 6 months we've been hard at work "retooling" the 3D building pipeline. We use the phrase "3D pipeline" to describe the process a model goes through when it's been published to the Google 3D Warehouse (assuming it was marked "Google Earth-ready").

The now outdated process for reviewing and publishing your 3D building models to Google Earth's 3D Buildings layer involved a number of manual steps and was inconsistent in terms of release dates. Further, and more importantly, far too many models were getting "lost" in the pipeline and remained in an idle state for long periods of time. We're keenly aware of how frustrating it is to publish a model and wait for Google to review and publish it. So, late last year, we set off on a mission to resolve these issues.

While this may appear simple on the surface, I can assure you that it's not. You might be surprised at the number of steps a model goes through from your publishing of it, to the model appearing in Google Earth. We like to be as transparent about the process as we can, so here is a high-level overview of those steps for your inquiring minds:

• KMZ creation - The very first thing that we do when you publish a model to the 3D Warehouse is create a KMZ file. This allows you and others to download/view your model in Google Earth.
• Model review - Next, a team of people review the model to ensure that it adheres to our acceptance criteria (more about that later).
• Duplicate check - In this step, we identify where duplicate and/or overlapping models exist. There's more of these than you might imagine, like 163 Eiffel Towers for example. This is also where we need to make decisions about which model is the best, and why some folks see a "better model exists" appear as a judgment on their model detail page . You'll be happy to know that when a user-provided model exists that is equal to or better than an auto-generated model, we will always choose the user model.
• Test build - After checking for duplicates, we "ingest" all of the data that has passed through the previous step to a test server where it receives another review. A small number of models will actually fail in this step for a few different reasons - it may be too complex for example, or have bad textures. While the evaluations prior to this point in the review process have been visual, this step checks for issues that might effect display, performance, or both. It's also where we make sure that cities are where they belong, that no space elevators shoot into space, and no Godzillas appear in Tokyo.
• Build and push - Finally, we push the models into our data centers so that they appear in your Google Earth client. Our data centers are spread out around the world so that, wherever you are, you can have a good experience in Google Earth.

What does all this mean to you? The new, automated, 3D pipeline will allow us to review and publish your models faster, and with far greater predictability. Specifically, the new system will enable us publish data on a *weekly* basis and, in the near future, even faster. So, with the exception of the one week per month where we need to conduct maintenance, we plan to publish models each and every week going forward.

Lost Models
Prior to this effort it was far too easy to find a model in the 3D Warehouse that appeared to be in a perpetual state of "awaiting review." Some models have been in this state for months or even years (ouch). So, in addition to retooling the process, we also scrubbed the 3D Warehouse for each and every geo-located model and sent it through our review process again. We scrubbed, and scrubbed some more, to make sure that every model that deserved to be in Google Earth was in Google Earth. Some of you noticed that a large number of models appeared in the layer in May because of this change. In fact, since we began this effort, we've *doubled* the number of user-generated models that appear in Google Earth today.

At this point we've completed our scrub and we're confident that every geo-located model has been reviewed. OK, full-disclosure.... there are a number of models that, for a variety of reasons, aren't added to the 3D Buildings layer *after* they have passed through our initial review (see "Test build" above). They look great visually, but have some other issue that may be preventing them from being successfully ingested into the layer. But, if you happen to own one of these models, fear not, we're working to identify what may be causing this. At a minimum, we'll want to update the feedback mechanism to make you aware that there's an issue with the model that's preventing it from appearing in Google Earth.

Acceptance Criteria
While some models have been overlooked, many others have been judged more rigorously than we'd like, so we modified our acceptance criteria to be a bit more lenient than it had been previously. We also developed and documented training material to ensure that we were applying a consistent approach in reviewing your models. It's still imperfect but I'm confident that it's far less subjective than it may have been before. As noted previously, we then ran all models that had been rejected through the pipeline again.

"So how has the acceptance criteria changed?" you ask. Good question. We plan to introduce some sample collections very soon that will serve as examples for what we deem acceptable, and highlight how to improve models that may need a little more work. It's not an easy job and it's one that we take very seriously. We're constantly balancing our desire to include as many models as possible, with an effort to establish a level of quality where, when combined, all of the 3D building data is visually harmonious. Geo-modeling is quite different than traditional SketchUp modeling, and we're seeing an increasing number of really fantastic phototextured models being published. Geo-modeling is definitely catching on!

In closing, we have several other features that we'll share with you soon, but suffice it to say that we've heard your pleas on the 3D Warehouse Group for faster turnaround and greater transparency in our process; we hope that you'll be pleased with the steps we've taken thus far.

Posted by Bruce Polderman, on behalf of the 3D Pipeline team

Permalink | Links to this post |