Apache MRQL

MRQL (pronounced miracle) is a query processing and optimization system for large-scale, distributed data analysis, built on top of Apache Hadoop, Hama, Spark, and Flink. Currently being incubated as one of the incubator project by the Apache Software Foundation.

MRQL (the MapReduce Query Language) is an SQL-like query language for large-scale data analysis on a cluster of computers. The MRQL query processing system can evaluate MRQL queries in 4 modes:

  1. in Map-Reduce mode using Apache Hadoop,
    2. in BSP mode (Bulk Synchronous Parallel mode) using Apache Hama,
    3. in Spark mode using Apache Spark, and
    4. in Flink mode using Apache Flink.

The MRQL query language is powerful enough to express most common data analysis tasks over many forms of raw in-situ data, such as XML and JSON documents, binary files, and CSV documents. MRQL is more powerful than other current high-level MapReduce languages, such as Hive and PigLatin, since it can operate on more complex data and supports more powerful query constructs, thus eliminating the need for using explicit Java code. With MRQL, users are able to express complex data analysis tasks, such as PageRank, k-means clustering, matrix factorization, etc, using SQL-like queries exclusively, while the MRQL query processing system is able to compile these queries to efficient Java code.

Disclaimer: Apache MRQL is an effort undergoing incubation at The Apache Software Foundation (ASF) sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.

General Information

User Documentation

Developer Documentation

  • How to contribute to MRQL
  • Guide for MRQL Committers.

  • Roadmap, listing release plans
    {{{#!html
    <script>
    (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]|function(){
    (i[r].q=i[r].q|[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
    m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
    })(window,document,'script','//www.google-analytics.com/analytics.js','ga');

ga('create', 'UA-62083383-1', 'auto');
ga('send', 'pageview');

</script>
}}}

  • No labels