Finding much faster ways to complete Hadoop queries for enterprise users is the aim of "Drill," the latest open-source project being undertaken by the Apache Software Foundation.
Drill has been established as an Apache Incubator Project, opening its continued development up to software engineers around the world, according to Tomer Shiran, director of product management for Hadoop vendor, MapR Technologies, which is one of the backers of the Apache Drill project.
The Drill project will work to create an open-source version of Google's DremelHadoop tools, which Google uses to speed up its internal use of its Hadoop data analysis tools.
"We've spent quite a few months talking to lots of organizations and potential users of Drill and to our customer base as well," said Shiran, who is a founding member of the Drill project. "We wanted to put this out there as an open-source project, rather than just keep it within MapR for our use alone."
Drill aids Hadoop users by enabling vastly quicker queries of huge data sets, said Shiran.
"With Drill, you'll be able to get really fast responses," he said. Users will be able to get responses within one second, which is a key difference from other tools that are available today, he added.
As it presently works as it was designed, Hadoop does batch processing of large data sets. Drill will improve on that method by doing "interactive analysis" that can find the required answers in the data more quickly, said Shiran. "Interactive analysis is much faster than batch processing."
The need for tools like Drill has been inspired by always-increasing user requirements, he said. "People have been doing queries in Hadoop, but since it doesn’t return answers to you within a few seconds, it has limitations."
Using Drill, users will be able to do ad hoc analysis and get faster responses, whether they are seeking anomalies, data trends or even network intrusions, according to Shiran. "With all of those things, you're going to have to get a pretty fast response or by the time you do figure it out, it's going to be old news."
The nascent Drill open-source project is currently in development and includes a variety of companies and individuals who are working on it right now. "A broad-based effort will be working on this," said Shiran. "There's quite a few people actively developing on the project now, so I don't think it will be a long time before we have an early version released."
Drill was inspired by Google's Dremel project, which helps Google perform data analyses on its huge data sets such as analyzing crawled Web documents, tracking install data for applications on the Android Market, analyzing spam, analyzing test results on Google’s distributed build system and more, according to Shiran.
By developing Drill as an Apache open-source project, organizers will be able to establish Drill's own APIs and establish a flexible and robust architecture that will support a broad range of data sources, data formats and query languages, according to the group.
MapR offers two versions of its Hadoop products: MapR M3, which is free; and MapR M5, which is a commercial version of the product with advanced features, including high availability, the ability to make data snapshots and do mirroring of datasets, and 24/7 support.
http://www.eweek.com/c/a/Application-Development/New-Apache-Project-Drill-Aims-to-Speed-Up-Hadoop-Queries-332333/
相关推荐
https://www.tutorialspoint.com/apache_drill/apache_drill_pdf_version.htm https://www.tutorialspoint.com/apache_drill/apache_drill_pdf_version.htm ...
使用Apache Drill技术
基于1.18版本整理了Apache Drill SQL中的常用函数
Learning Apache Drill 2019 最新版
Learning Apache Drill
数据整合处理的工具,apache-drill
Learning Apache Drill Queryand Analyze Distributed Data Sources with SQL
apache-drill-jdbc-plugin 适用于Apache Drill的JDBC插件 下载Apache Drill 0.9。 将代码添加到contrib中,然后用此文件夹中的pom文件替换现有的pom文件。 用mvn构建。 要仅生成软件包,请使用与以下类似的符号:...
介绍Apache Kylin的背景,技术架构及演进,产品功能和性能等内容,以及开源现状和发展方向等,例如与Spark/Drill等的集成等。Apache Kylin是由eBay研发并贡献给开源社区的Hadoop上的分布式大规模联机分析(OLAP)...
第一部关于Apache Drill的技术文档,该文档属于作者自行整理文档资料,如果误差请谅解。
Apache Drill是一个分布式MPP查询层,支持针对NoSQL和Hadoop数据存储系统SQL和替代查询语言。 它的部分灵感来自 。 开发者 请阅读以设置和运行Apache Drill。 有关完整的开发人员文档,请参见 更多信息 请参阅或以...
一组用于处理Internet域名的Apache Drill UDF UDFs 有一个UDF: suffix_extract(domain-string) :给定一个有效的互联网域名(FQDN或其他方式),这将返回一个地图的领域tld , assigned , subdomain和hostname的...
核心模块下列流程图呈现的是 Drillbit 的各个组件:以下是描述一个 Drillbit 关键组件的清单,如下所示:RPC endpoint:Drill 提供
用于 Apache Drill 的 Ruby 客户端 安装 首先, 。 对于 Homebrew,请使用: brew install apache-drill drill-embedded 并将这一行添加到您的应用程序的 Gemfile 中: gem 'drill-sergeant' 如何使用 创建...
Apache Drill的Node.js客户端Drillnode是使用REST API从Node.js连接并在Apache Drill上执行查询的客户端。安装使用NPM软件包管理器安装Drillone npm install drillnode入门// Requirementsconst Drill = require ( '...
Apache Drill UDF用于检索和使用HTML文本 基于库。 注意:这绝对是一个在制品。 UDFs soup_read_html(url-string, timeout-ms) :此UDF要求网络可到达预期的URL目标。 给定一个URL和一个连接超时(以毫秒为单位)...
一个Apache Drill UDF,用于通过 Java库处理Twitter tweet文本。 UDFs tw_parse_tweet(string) :解析tweet文本并返回具有以下命名值的地图列: weightedLength :(整数)tweet的总长度,其中代码点按配置文件中...
很好的学习了解Drill up/down的资料。
apache-drill-函数 掩码示例 0: jdbc:drill:> 从dfs中选择account_id、amount、transaction_id。 /frdo/gess/json/demo_gess_sink.json其中 account_id LIKE 'a61%' and amount > 50 order by transaction_id limit ...
军士:使用“ Apache”“ Drill”转换和查询数据的工具