I'm not a technologist. Part of the reason I was recruited by an old college classmate of mine to join MarkLogic is because of my perspective on missions, and what the intelligence and defence communities do in their mission with respect to the use of intelligence. I tell people at my talks that my mission for the last 30 years has been making sure that commanders and policymakers have the information they need to make intelligent, informed decisions.
 
MarkLogic makes data available to people at speed so that they can make use of the collected information at a time when the volume of data collected and information available has just exploded. It's almost completely beyond the capacity of a single human to follow all of this, so they need tools, and what MarkLogic does mirrors what I did in my career.

Not enough! There are four areas we really excel at in the national security realm. One is logistics management, the other is person management, the third is records management, and the fourth is data analytics and the deep study of data collected.

We have what we call an operational data hub. In those areas, we are assisting two American intelligence organisations with their records management and the ability to establish relationships amongst suspicious personnel, networks and organisations. We were able to tie together disparate facts and learn more about networks than they did before based on the data that we collect, assemble and assist with.

For the military we also maintain messaging and the ability to record and establish all the various records of their messaging systems, which can number in the millions; a very tall order when you have tens of thousands of users writing tens of thousands of messages each day.

Another area where we do a lot of work is material research. We are the backbone for the US Air Force Research Laboratory and the archives of their experimentation, results and materials, so they are available across domains and laboratories.

We've got legacy systems going back so long, with relational models in rows and columns, that that model no longer fits with the modern analytic capabilities and aspirations. We have our current analytic capabilities, as well as where we want to go in the future, and those rows and columns are not going to answer those requirements.

The nice thing about MarkLogic, and in fact any multi-model NoSQL database, is the way we bring information in. We accept as it is; we don't have to do a lot of extraction and transformation loading processes, there's no ETL [extract, transform, load]. Everything we ingest, we ingest as a document, and it's immediately indexed, immediately searchable.

We record every transaction associated with that document so that you know who used it, when they used it and how they used it; any changes are recorded across the entire stack. That metadata associated with that file is critical for machine learning, deep learning and artificial intelligence to be able to make statements about how the data was governed and to give faith and credit and competence that the data supplied is pristine, not corrupted, [and] secured, but available.

It's got to be shared; security isn't about safeguarding stuff, you can’t put things in a vault, close it, weld it shut, no one ever gets in. Data has to be accessible.

MarkLogic allows you to bring in that data, keep it well-governed, make statements about its integrity, and then give it back to the users in the format in which it arrives. We're really good with JSON and XML files. We can handle anything, even to the point where we're figuring out the full-motion video problem set, where the volume of those files around is so enormous.

We are working with our partners on a C5ISR hub that will enable the customer to monitor and observe ISR feeds, searching reconnaissance feeds for activity-based indicators. We are teaching the algorithms to look for ships in port and what changes to the profile in a major adversary’s base port look like. When food and fuel trucks pull up onto the pier and load up, the ship is planning to deploy. They’re looking at airfields where the order of battle for an airfield is known, and detecting changes in activity.

One great altruistic programme we do is with a Dutch non-government organisation, Sensing Clues, to help protect endangered species. We work with park rangers in the national parks by monitoring the input from various sensors that track the whereabouts of the animals and monitor the environment around them to detect poaching activity and then maintain a profile of where the park rangers themselves are patrolling to alert them to respond to potential poaching activity.