aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2018-11-15DRILL-6744: Support varchar and decimal push downArina Ielchiieva
1. Added enableStringsSignedMinMax parquet format plugin config and store.parquet.reader.strings_signed_min_max session option to control reading binary statistics for files generated by prior versions of Parquet 1.10.0. 2. Added ParquetReaderConfig to store configuration needed during reading parquet statistics or files. 3. Provided mechanism to enable varchar / decimal filter push down. 4. Added VersionUtil to compare Drill versions in string representation. 5. Added appropriate unit tests. closes #1537
2018-11-15DRILL-3933: Surround $QUERY variable in double-quotes to avoid asterisk ↵Arina Ielchiieva
expansion in sqlline script closes #1538
2018-11-15DRILL-5671: Set secure ACLs (Access Control List) for Drill ZK nodes in a ↵karthik
secure cluster closes #1467
2018-11-15DRILL-6843: Update SchemaBuilder comment to match implementationPaul Rogers
The schema builder API evolved a while back. The class provides a detailed comment to explain usage. That comment was out of date. This fix updates the comment to match the code. closes #1533
2018-11-15DRILL-786: Allow CROSS JOIN syntaxIgor Guzenko
1. Removed throw statement in UnsupportedOperatorsVisitor 2. Extended UnsupportedRelOperatorException's message closes #1488
2018-11-13DRILL-6848: Duration panel in a query profile's WebUI does not openKunal Khatua
DRILL-5571 ( PR #1531 ) accidentally introduced a bug that permanently keeps the "Duration" panel closed instead of opening it. This trivial patch fixes that.
2018-11-13Drill 6735: Implement Semi-Join for the Hash-Join operator (#1522)Boaz Ben-Zvi
2018-11-13DRILL-2035: Add ability to cancel multiple queriesKunal Khatua
Currently Drill UI allows canceling one query at a time. This commit (on lines of DRILL-5571 / PR #1531) allows for cancelling multiple `running` queries.
2018-11-13DRILL-5571: Cancel running query from its Web UIKunal Khatua
Currently, a running query can be cancelled by navigating to the Edit Query Tab of the profile page, making it difficult to access. This commit allows the user to cancel. In addition, the Duration segment is also made visible by default rather than collapsed.
2018-11-09DRILL-6760: Retain original exception in Verbose Error Messageshuifeng lu
closes #1519
2018-11-09DRILL-6717: lower and upper functions not works with national charactersozinoviev
closes #1450
2018-11-09DRILL-6610: Add support for Minimum TLS restriction.superbstreak
closes #1388
2018-11-09DRILL-4456: Add Hive translate UDFVolodymyr Vysotskyi
closes #1527
2018-11-09DRILL-3610: Add TIMESTAMPADD and TIMESTAMPDIFF functionsVolodymyr Vysotskyi
closes #1528
2018-11-09DRILL-6837: Missing reference for dynamically created Javascript libraryKunal Khatua
A PR for DRILL-6084 broke the auto-complete library required for Edit Query page, with the removal of the static javascript library. https://github.com/apache/drill/pull/1491/files#diff-2bb3c04f1a9eead1faf57212cce5d2a9 The dynamically loaded script is not defined in the Freemarker template page. The fix is trivial.
2018-11-05DRILL-6611: Add Ctrl+Enter support for query submissionKunal Khatua
1. BugFix on parent commit: Ensure query submission is done with user name when impersonation is enabled. 2. Support non-Mac browsers 3. Support keyboard submission for profile pages with Edit Query tab.
2018-11-05DRILL-6611 to enable meta-enter query submission in web query interfaceboB Rudis
2018-11-02DRILL-6768: Improve to_date, to_time and to_timestamp and corresponding cast ↵Bohdan Kazydub
functions to handle empty string when option is enabled closes #1494
2018-11-02DRILL-6809: Handle repeated map in schema inferencePaul Rogers
It turns out that the RowSet utilities build a repeated map without including the hidden $offsets$ vector in the metadata for the map. But, other parts in Drill do include this vector. The RowSet behavior might be a bug which can be addressed in another PR. This PR: * Adds unit tests for map accessors at the row set level. Looks like these were never added originally. They are a simplified form of the ResultSetLoader map tests. * Verified that the schema inference can infer a schema from a repeated map (using the RowSet style.) * Added a test to reproduce the case from the bug. * Made a tweak to the RowSetBuilder to allow access to the RowSetWriter which is needed by the new tests. * Could of minor clean-ups. closes #1513
2018-11-02DRILL-6810: Disable NULL_IF_NULL NullHandling for functions with ComplexWriterBohdan Kazydub
closes #1509
2018-11-02DRILL-6824: Handle schema changes in MapRDBJsonRecordReaderGautam Parai
closes #1518
2018-11-01DRILL-6798: Planner changes to support semi-join.Hanumath Rao Maduri
2018-11-01DRILL-6819: Remove invisible back link in Drill WebUIDenys Ordynskiy
2018-11-01Fixed imports for DRILL-6381dvjyothsna
2018-10-26DRILL-6763: Codegen optimization of SQL functions with constant values(#1481)shuifeng lu
closes #1481
2018-10-26DRILL-6811: Fix type inference to return correct data mode for boolean functionsVolodymyr Vysotskyi
closes #1510
2018-10-25DRILL-6381: Address code review comments (part 3).Aman Sinha
DRILL-6381: Add missing joinControl logic for INTERSECT_DISTINCT. - Modified HashJoin's probe phase to process INTERSECT_DISTINCT. - NOTE: For build phase, the functionality will be same as for SemiJoin when it is added later. DRILL-6381: Address code review comment for intersect_distinct. DRILL-6381: Rebase on latest master and fix compilation issues. DRILL-6381: Generate protobuf files for C++ native client. DRILL-6381: Use shaded Guava classes. Add more comments and Javadoc.
2018-10-25DRILL-6381: Address review comments (part 2): fix formatting issues and add ↵Aman Sinha
javadoc.
2018-10-25DRILL-6381: Address code review comments.Aman Sinha
2018-10-25DRILL-6381: (Part 5) Update Javadoc for a few interfaces.Aman Sinha
2018-10-25DRILL-6381: (Part 4) Enhance MapR-DB plugin to support querying secondary ↵rebase
indexes   1. Implementation of the index descriptor for MapR-DB. 2. MapR-DB specific costing for covering and non-covering indexes. 3. Discovery componenent to discover the indexes available for a MapR-DB table including CAST functional indexes. 4. Utility functions to build a canonical index descriptor. 5. Statistics: fetch and initialize statistcs from MapR-DB for a query condition. Maintain a query-scoped cache for the statistics. Utility functions to compute selectivity. 6. Range Partitioning: partitioning function that takes into account the tablet map to find out where a particular rowkey belongs. 7. Restricted Scan: support doing restricted (i.e skip) scan through lookups on the rowkey. Added a group-scan and record reader for this. 8. MD-3726: Simple Order by queries (without limit) when an index is used are showing regression. 9. MD-3995: Do not pushdown limit 0 past project with CONVERT_FROMJSON 10. MD-4259 : Account for limit during hashcode computation Co-authored-by: Aman Sinha <asinha@maprtech.com> Co-authored-by: chunhui-shi <cshi@maprtech.com> Co-authored-by: Gautam Parai <gparai@maprtech.com> Co-authored-by: Padma Penumarthy <ppenumar97@yahoo.com> Co-authored-by: Hanumath Rao Maduri <hmaduri@maprtech.com> Conflicts: contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/MapRDBFormatMatcher.java contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/MapRDBPushProjectIntoScan.java contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/JsonTableGroupScan.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/index/rules/DbScanSortRemovalRule.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/SortPrel.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/TopNPrel.java Fix additional compilation issues.
2018-10-25DRILL-6381: (Part 3) Planner and Execution implementation to support ↵rebase
Secondary Indexes   1. Index Planning Rules and Plan generators     - DbScanToIndexScanRule: Top level physical planning rule that drives index planning for several relational algebra patterns. - DbScanSortRemovalRule: Physical planning rule for index planning for Sort-based operations.     - Plan Generators: Covering, Non-Covering and Intersect physical plan generators.     - Support planning with functional indexes such as CAST functions.     - Enhance PlannerSettings with several configuration options for indexes.   2. Index Selection and Statistics     - An IndexSelector that support cost-based index selection of covering and non-covering indexes using statistics and collation properties.     - Costing of index intersection for comparison with single-index plans.   3. Planning and execution operators     - Support RangePartitioning physical operator during query planning and execution.     - Support RowKeyJoin physical operator during query planning and execution.     - HashTable and HashJoin changes to support RowKeyJoin and Index Intersection.     - Enhance Materializer to keep track of subscan association with a particular rowkey join.   4. Index Planning utilities     - Utility classes to perform RexNode analysis, including conversion to and from SchemaPath.     - Utility class to analyze filter condition and an input collation to determine output collation.     - Helper classes to maintain index contexts for logical and physical planning phase.     - IndexPlanUtils utility class for various helper methods.   5. Miscellaneous     - Separate physical rel for DirectScan.     - Modify LimitExchangeTranspose rule to handle SingleMergeExchange. - MD-3880: Return correct status from RangePartitionRecordBatch setupNewSchema Co-authored-by: Aman Sinha <asinha@maprtech.com> Co-authored-by: chunhui-shi <cshi@maprtech.com> Co-authored-by: Gautam Parai <gparai@maprtech.com> Co-authored-by: Padma Penumarthy <ppenumar97@yahoo.com> Co-authored-by: Hanumath Rao Maduri <hmaduri@maprtech.com> Conflicts: exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashJoinPOP.java exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashPartition.java exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTable.java exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTableTemplate.java exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillRelOptUtil.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Materializer.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillMergeProjectRule.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillOptiq.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillPushProjectIntoScanRule.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillScanRel.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/BroadcastExchangePrel.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/DrillDistributionTrait.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashJoinPrel.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PrelUtil.java exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetPushDownFilter.java exec/java-exec/src/main/resources/drill-module.conf logical/src/main/java/org/apache/drill/common/logical/StoragePluginConfig.java Resolve merge comflicts and compilation issues.
2018-10-25DRILL-6381: (Part 2) MapRDB plugin update to 6.0.1rebase
1. MD-3960: Update Drill to build with MapR-6.0.1 libraries   2. MD-3995: Do not pushdown limit 0 past project with CONVERT_FROMJSON 3. MD-4054: Restricted scan limit is changed to dynamically read rows using the rowcount of the rightside instead of 4096. 4. MD-3688: Impersonating a view owner doesn't work with security disabled in 6.0 5. MD-4492: Missing limit pushdown changes in JsonTableGroupScan Co-authored-by: chunhui-shi <cshi@maprtech.com> Co-authored-by: Gautam Parai <gparai@maprtech.com> Co-authored-by: Vlad Rozov <vrozov@mapr.com> Conflicts: contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/MapRDBFormatPlugin.java contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/MapRDBSubScan.java contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/binary/BinaryTableGroupScan.java contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/CompareFunctionsProcessor.java contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/JsonConditionBuilder.java contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/JsonTableGroupScan.java contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/MaprDBJsonRecordReader.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PrelUtil.java pom.xml
2018-10-25DRILL-6381: (Part 1) Secondary Index frameworkrebase
  1. Secondary Index planning interfaces and abstract classes like DBGroupScan, DbSubScan, IndexDecriptor etc.   2. Statistics and Cost model interfaces/classes: PluginCost, Statistics, StatisticsPayload, AbstractIndexStatistics   3. ScanBatch and RecordReader to support repeatable scan   4. Secondary Index execution related interfaces: RangePartitionSender, RowKeyJoin, PartitionFunction 5. MD-3979: Query using cast index plan fails with NPE Co-authored-by: Aman Sinha <asinha@maprtech.com> Co-authored-by: chunhui-shi <cshi@maprtech.com> Co-authored-by: Gautam Parai <gparai@maprtech.com> Co-authored-by: Padma Penumarthy <ppenumar97@yahoo.com> Co-authored-by: Hanumath Rao Maduri <hmaduri@maprtech.com> Conflicts: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillRelOptUtil.java exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillTable.java protocol/src/main/java/org/apache/drill/exec/proto/UserBitShared.java protocol/src/main/java/org/apache/drill/exec/proto/beans/CoreOperatorType.java protocol/src/main/protobuf/UserBitShared.proto
2018-10-25DRILL-6715: Update descriptions for System Options tableKunal Khatua
With introduction of DRILL-5735 , the descriptions for about half the system options still remain missing. This commit collects descriptions review by @bbevens 1. Update options for HashAgg/Join (@Ben-Zvi ) 2. Update options for Parquet Reader/Writer (@sachouche ) 3. Update options for Planners (@HanumathRao , @vdiravka , @KazydubB ) 4. Update options for BatchSizing (@bitblender ) 5. Update options for Planner Optimizations (@arina-ielchiieva ) 6. Update options for Security & Kafka (Krystal Nguyen) 7. Update options for Misc entries (@arina-ielchiieva , @vvysotskyi ) In additional, there is a patch for `org.apache.drill.exec.compile.ClassTransformer.scalar_replacement` , which appears to have replaced `exec.compile.scalar_replacement`. References to the latter have been removed to avoid confusion. Additional changes include moving the `ClassTransformer` validator to `ExecConstants.java` Adding support for internal options' descriptions Removed mention of {{Will be removed in 1.15.0}}. (Refer DRILL-6527)
2018-10-23DRILL-6804: Simplify usage of OperatorPhase in HashAgg.Timothy Farkas
2018-10-19DRILL-6719: Separate spilling queue logic from HashJoin and HashAgg.Timothy Farkas
2018-10-19DRILL-6795: Upgrade Janino compiler from 2.7.6 to 3.0.10Volodymyr Vysotskyi
closes #1503
2018-10-19DRILL-6793: FragmentExecutor cannot send its final state for the case when ↵Bohdan Kazydub
RootExec root wasn't initialized closes #1506
2018-10-19DRILL-6785: DataClient is using RootAllocator in the bootstrap instead of ↵Sorabh Hamirwasia
dataPool closes #1502
2018-10-17DRILL-6084: Show Drill functions in WebUI for autocompleteKunal Khatua
Building on top of DRILL-3988 and leveraging DRILL-5868, this allows support for Drill functions to be now available in the WebUI. If users wants UDFs to show up, they should place the UDF jars in the `$DRILL_HOME/jars/3rdparty` directory so that this can be loaded during the Drillbit's startup. Concept of internal Drill functions are introduced. With this, internal Drill functions like `ConvertToNullableXYZ` has been marked as internal. The WebUI will not show these functions. However, they are still visible in `sys.functions` table with an additional column indicating that it is an internal function. Tests have been added as a part of this commit to verify the internal functions concept.
2018-10-17DRILL-6797: Fix UntypedNull handling for complex typesArina Ielchiieva
2018-10-14DRILL-6775: The schema for empty output is not shown in Drill Web UIAnton Gozhiy
Removed an excess check for the result emptiness that prevented retrieval of the column names. close apache/drill#1498
2018-10-14DRILL-6783: CAST string literal as INTERVAL MONTH/YEAR works inconsistently ↵Bohdan Kazydub
when selecting from a table with multiple rows close apache/drill#1496
2018-10-14DRILL-6777: Setup CircleCI configs for DrillVitalii Diravka
- adding .circleci/config.yml to the project to launch CircleCI - custom memory parameters - usage of CircleCI machine - excluding "SlowTest" and "UnlikelyTest" groups - update maven version - adding libaio.so library to solve MySQL integration tests - update com.jcabi:jcabi-mysql-maven-plugin library version - TODO descriptions for the future enhancements of CircleCI build for Drill close apache/drill#1493
2018-10-14DRILL-6473: Update MapR HiveBohdan Kazydub
close apache/drill#1307
2018-10-11DRILL-3988: Expose Drill built-in functions & UDFs in a system table (#1483)Kunal Khatua
This commit exposes available SQL functions in Drill and also detects UDFs that have been dynamically loaded into Drill. An example is shown below for 2 UDFs dynamically loaded into the cluster, along side the existing built-in functions that come with Drill. ``` 0: jdbc:drill:schema=sys> select source, count(*) as functionCount from sys.functions group by source; +-----------------------------------------+----------------+ | source | functionCount | +-----------------------------------------+----------------+ | built-in | 2704 | | simple-drill-function-1.0-SNAPSHOT.jar | 12 | | drill-url-tools-1.0.jar | 1 | +-----------------------------------------+----------------+ 3 rows selected (0.209 seconds) ``` The system table exposes information as shown. The UDF is initialized, making the `returnType` available. The `random(FLOAT8-REQUIRED,FLOAT8-REQUIRED)` function is an example of a UDF that has overloaded arguments (see `signature`). The `url_parse(VARCHAR-REQUIRED)` function is another example of an initialized UDF. Rest are built-in functions that meet the query's filter criteria. ``` 0: jdbc:drill:schema=sys> select * from sys.functions where name like 'random' or name like '%url%'; +-------------+----------------------------------+-------------+-----------------------------------------+ | name | signature | returnType | source | +-------------+----------------------------------+-------------+-----------------------------------------+ | parse_url | VARCHAR-REQUIRED | LATE | built-in | | random | | FLOAT8 | built-in | | random | FLOAT8-REQUIRED,FLOAT8-REQUIRED | FLOAT8 | simple-drill-function-1.0-SNAPSHOT.jar | | url_decode | VARCHAR-REQUIRED | VARCHAR | built-in | | url_encode | VARCHAR-REQUIRED | VARCHAR | built-in | | url_parse | VARCHAR-REQUIRED | LATE | drill-url-tools-1.0.jar | +-------------+----------------------------------+-------------+-----------------------------------------+ 6 rows selected (0.619 seconds) ```
2018-10-11DRILL-6776: Removed external jQuery dependencyIgor Guzenko
2018-10-10DRILL-6731: use thread pool to run the runtime filter aggregating workweijie.tong
closes #1459
2018-10-10DRILL-6731: Resolving race conditions in RuntimeFilterSinkSorabh Hamirwasia
Add condition variable to avoid starvation of producer thread while acquiring queue lock
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-10 00:39:53 +0000