diff options
Diffstat (limited to 'bigtop-packages/src/charm/hadoop/layer-hadoop-plugin/README.md')
| -rw-r--r-- | bigtop-packages/src/charm/hadoop/layer-hadoop-plugin/README.md | 100 |
1 files changed, 65 insertions, 35 deletions
diff --git a/bigtop-packages/src/charm/hadoop/layer-hadoop-plugin/README.md b/bigtop-packages/src/charm/hadoop/layer-hadoop-plugin/README.md index cbea7f02..405c08ac 100644 --- a/bigtop-packages/src/charm/hadoop/layer-hadoop-plugin/README.md +++ b/bigtop-packages/src/charm/hadoop/layer-hadoop-plugin/README.md @@ -14,79 +14,109 @@ See the License for the specific language governing permissions and limitations under the License. --> -## Overview +# Overview The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using a simple programming model. -This charm facilitates communication between core Apache Bigtop cluster -components and workload charms. +This charm facilitates communication between Hadoop components of an +[Apache Bigtop][] cluster and workload applications. +[Apache Bigtop]: http://bigtop.apache.org/ -## Usage -This charm is intended to be deployed via one of the -[apache bigtop bundles](https://jujucharms.com/u/bigdata-dev/#bundles). -For example: +# Deploying - juju deploy hadoop-processing +A working Juju installation is assumed to be present. If Juju is not yet set +up, please follow the [getting-started][] instructions prior to deploying this +charm. -> Note: With Juju versions < 2.0, you will need to use [juju-deployer][] to -deploy the bundle. +This charm is intended to be deployed via one of the [apache bigtop bundles][]. +For example: -This will deploy the Apache Bigtop platform with a workload node -preconfigured to work with the cluster. + juju deploy hadoop-processing -You could extend this deployment, for example, to analyze data using Apache Pig. -Simply deploy Pig and attach it to the same plugin: +> **Note**: The above assumes Juju 2.0 or greater. If using an earlier version +of Juju, use [juju-quickstart][] with the following syntax: `juju quickstart +hadoop-processing`. - juju deploy apache-pig pig - juju add-relation plugin pig +This will deploy an Apache Bigtop cluster with a client unit preconfigured to +work with the cluster. More information about this deployment can be found in the +[bundle readme](https://jujucharms.com/hadoop-processing/). +## Network-Restricted Environments +Charms can be deployed in environments with limited network access. To deploy +in this environment, configure a Juju model with appropriate proxy and/or +mirror options. See [Configuring Models][] for more information. -[juju-deployer]: https://pypi.python.org/pypi/juju-deployer/ +[getting-started]: https://jujucharms.com/docs/stable/getting-started +[apache bigtop bundles]: https://jujucharms.com/u/bigdata-charmers/#bundles +[juju-quickstart]: https://launchpad.net/juju-quickstart +[Configuring Models]: https://jujucharms.com/docs/stable/models-config -## Status and Smoke Test +# Verifying +## Status Apache Bigtop charms provide extended status reporting to indicate when they are ready: - juju status --format=tabular + juju status This is particularly useful when combined with `watch` to track the on-going progress of the deployment: - watch -n 0.5 juju status --format=tabular + watch -n 2 juju status + +The message column will provide information about a given unit's state. +This charm is ready for use once the status message indicates that it is +ready with hdfs and/or yarn. + +## Smoke Test +This charm provides a `smoke-test` action that can be used to verify the +application is functioning as expected. Run the action as follows: + + juju run-action plugin/0 smoke-test + +> **Note**: The above assumes Juju 2.0 or greater. If using an earlier version +of Juju, the syntax is `juju action do plugin/0 smoke-test`. + +Watch the progress of the smoke test actions with: + + watch -n 2 juju show-action-status + +> **Note**: The above assumes Juju 2.0 or greater. If using an earlier version +of Juju, the syntax is `juju action status`. -The message for each unit will provide information about that unit's state. -Once they all indicate that they are ready, you can perform a "smoke test" -to verify HDFS or YARN services are working as expected. Trigger the -`smoke-test` action by: +Eventually, the action should settle to `status: completed`. If it +reports `status: failed`, the application is not working as expected. Get +more information about a specific smoke test with: - juju action do namenode/0 smoke-test - juju action do resourcemanager/0 smoke-test + juju show-action-output <action-id> -After a few seconds or so, you can check the results of the smoke test: +> **Note**: The above assumes Juju 2.0 or greater. If using an earlier version +of Juju, the syntax is `juju action fetch <action-id>`. - juju action status +## Utilities +This charm includes Hadoop command line utilities that can be used +to verify information about the cluster. -You will see `status: completed` if the smoke test was successful, or -`status: failed` if it was not. You can get more information on why it failed -via: +Show the dfsadmin report on the command line with the following: - juju action fetch <action-id> + juju run --application plugin "su hdfs -c 'hdfs dfsadmin -report'" -## Contact Information +# Contact Information - <bigdata@lists.ubuntu.com> -## Resources +# Resources - [Apache Bigtop](http://bigtop.apache.org/) home page - [Apache Bigtop issue tracking](http://bigtop.apache.org/issue-tracking.html) - [Apache Bigtop mailing lists](http://bigtop.apache.org/mail-lists.html) -- [Apache Bigtop charms](https://jujucharms.com/q/apache/bigtop) +- [Juju Bigtop charms](https://jujucharms.com/q/apache/bigtop) +- [Juju mailing list](https://lists.ubuntu.com/mailman/listinfo/juju) +- [Juju community](https://jujucharms.com/community) |