aboutsummaryrefslogtreecommitdiff
path: root/bigtop-deploy/juju/hadoop-kafka
diff options
context:
space:
mode:
Diffstat (limited to 'bigtop-deploy/juju/hadoop-kafka')
-rw-r--r--bigtop-deploy/juju/hadoop-kafka/README.md49
-rw-r--r--bigtop-deploy/juju/hadoop-kafka/bundle-dev.yaml15
-rw-r--r--bigtop-deploy/juju/hadoop-kafka/bundle-local.yaml15
-rw-r--r--bigtop-deploy/juju/hadoop-kafka/bundle.yaml27
-rwxr-xr-xbigtop-deploy/juju/hadoop-kafka/tests/01-bundle.py2
-rw-r--r--bigtop-deploy/juju/hadoop-kafka/tests/tests.yaml2
6 files changed, 64 insertions, 46 deletions
diff --git a/bigtop-deploy/juju/hadoop-kafka/README.md b/bigtop-deploy/juju/hadoop-kafka/README.md
index 4c7a5818..4caf7744 100644
--- a/bigtop-deploy/juju/hadoop-kafka/README.md
+++ b/bigtop-deploy/juju/hadoop-kafka/README.md
@@ -44,15 +44,15 @@ activity.
The applications that comprise this bundle are spread across 9 units as
follows:
- * NameNode (HDFS)
- * ResourceManager (YARN)
+ * NameNode v2.7.3
+ * ResourceManager v2.7.3
* Colocated on the NameNode unit
- * Slave (DataNode and NodeManager)
+ * Slave (DataNode and NodeManager) v2.7.3
* 3 separate units
- * Kafka
+ * Kafka v0.10.1
* Flume-Kafka
* Colocated on the Kafka unit
- * Zookeeper
+ * Zookeeper v3.4.6
* 3 separate units
* Client (Hadoop endpoint)
* Plugin (Facilitates communication with the Hadoop cluster)
@@ -76,9 +76,8 @@ demands.
# Deploying
-A working Juju installation is assumed to be present. If Juju is not yet set
-up, please follow the [getting-started][] instructions prior to deploying this
-bundle.
+This charm requires Juju 2.0 or greater. If Juju is not yet set up, please
+follow the [getting-started][] instructions prior to deploying this bundle.
> **Note**: This bundle requires hardware resources that may exceed limits
of Free-tier or Trial accounts on some clouds. To deploy to these
@@ -90,18 +89,10 @@ Deploy this bundle from the Juju charm store with the `juju deploy` command:
juju deploy hadoop-kafka
-> **Note**: The above assumes Juju 2.0 or greater. If using an earlier version
-of Juju, use [juju-quickstart][] with the following syntax: `juju quickstart
-hadoop-kafka`.
-
Alternatively, deploy a locally modified `bundle.yaml` with:
juju deploy /path/to/bundle.yaml
-> **Note**: The above assumes Juju 2.0 or greater. If using an earlier version
-of Juju, use [juju-quickstart][] with the following syntax: `juju quickstart
-/path/to/bundle.yaml`.
-
The charms in this bundle can also be built from their source layers in the
[Bigtop charm repository][]. See the [Bigtop charm README][] for instructions
on building and deploying these charms locally.
@@ -113,7 +104,6 @@ mirror options. See [Configuring Models][] for more information.
[getting-started]: https://jujucharms.com/docs/stable/getting-started
[bundle.yaml]: https://github.com/apache/bigtop/blob/master/bigtop-deploy/juju/hadoop-kafka/bundle.yaml
-[juju-quickstart]: https://launchpad.net/juju-quickstart
[Bigtop charm repository]: https://github.com/apache/bigtop/tree/master/bigtop-packages/src/charm
[Bigtop charm README]: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/charm/README.md
[Configuring Models]: https://jujucharms.com/docs/stable/models-config
@@ -166,25 +156,16 @@ complete. Run the smoke-test actions as follows:
juju run-action kafka/0 smoke-test
juju run-action zookeeper/0 smoke-test
-> **Note**: The above assumes Juju 2.0 or greater. If using an earlier version
-of Juju, the syntax is `juju action do <application>/0 smoke-test`.
-
Watch the progress of the smoke test actions with:
watch -n 2 juju show-action-status
-> **Note**: The above assumes Juju 2.0 or greater. If using an earlier version
-of Juju, the syntax is `juju action status`.
-
Eventually, all of the actions should settle to `status: completed`. If
any report `status: failed`, that application is not working as expected. Get
more information about a specific smoke test with:
juju show-action-output <action-id>
-> **Note**: The above assumes Juju 2.0 or greater. If using an earlier version
-of Juju, the syntax is `juju action fetch <action-id>`.
-
## Utilities
Applications in this bundle include command line and web utilities that
can be used to verify information about the cluster.
@@ -314,6 +295,18 @@ Multiple units may be added at once. For example, add four more slave units:
juju add-unit -n4 slave
+# Issues
+
+Apache Bigtop tracks issues using JIRA (Apache account required). File an
+issue for this bundle at:
+
+https://issues.apache.org/jira/secure/CreateIssue!default.jspa
+
+Ensure `Bigtop` is selected as the project. Typically, bundle issues are filed
+in the `deployment` component with the latest stable release selected as the
+affected version. Any uncertain fields may be left blank.
+
+
# Contact Information
- <bigdata@lists.ubuntu.com>
@@ -324,6 +317,6 @@ Multiple units may be added at once. For example, add four more slave units:
- [Apache Bigtop home page](http://bigtop.apache.org/)
- [Apache Bigtop issue tracking](http://bigtop.apache.org/issue-tracking.html)
- [Apache Bigtop mailing lists](http://bigtop.apache.org/mail-lists.html)
-- [Juju Bigtop charms](https://jujucharms.com/q/apache/bigtop)
+- [Juju Big Data](https://jujucharms.com/big-data)
+- [Juju Bigtop charms](https://jujucharms.com/q/bigtop)
- [Juju mailing list](https://lists.ubuntu.com/mailman/listinfo/juju)
-- [Juju community](https://jujucharms.com/community)
diff --git a/bigtop-deploy/juju/hadoop-kafka/bundle-dev.yaml b/bigtop-deploy/juju/hadoop-kafka/bundle-dev.yaml
index 45b821fa..f715f788 100644
--- a/bigtop-deploy/juju/hadoop-kafka/bundle-dev.yaml
+++ b/bigtop-deploy/juju/hadoop-kafka/bundle-dev.yaml
@@ -34,7 +34,7 @@ services:
gui-x: "1000"
gui-y: "400"
client:
- charm: "cs:xenial/hadoop-client-3"
+ charm: "cs:xenial/hadoop-client-8"
constraints: "mem=3G"
num_units: 1
annotations:
@@ -81,7 +81,7 @@ services:
to:
- "8"
ganglia:
- charm: "cs:~bigdata-dev/xenial/ganglia-5"
+ charm: "cs:xenial/ganglia-12"
num_units: 1
annotations:
gui-x: "0"
@@ -89,7 +89,7 @@ services:
to:
- "4"
ganglia-node:
- charm: "cs:~bigdata-dev/xenial/ganglia-node-7"
+ charm: "cs:xenial/ganglia-node-7"
annotations:
gui-x: "250"
gui-y: "400"
@@ -133,19 +133,28 @@ relations:
machines:
"0":
series: "xenial"
+ constraints: "mem=7G root-disk=32G"
"1":
series: "xenial"
+ constraints: "mem=7G root-disk=32G"
"2":
series: "xenial"
+ constraints: "mem=7G root-disk=32G"
"3":
series: "xenial"
+ constraints: "mem=7G root-disk=32G"
"4":
series: "xenial"
+ constraints: "mem=3G"
"5":
series: "xenial"
+ constraints: "mem=3G root-disk=32G"
"6":
series: "xenial"
+ constraints: "mem=3G root-disk=32G"
"7":
series: "xenial"
+ constraints: "mem=3G root-disk=32G"
"8":
series: "xenial"
+ constraints: "mem=3G"
diff --git a/bigtop-deploy/juju/hadoop-kafka/bundle-local.yaml b/bigtop-deploy/juju/hadoop-kafka/bundle-local.yaml
index bd898728..82bfb164 100644
--- a/bigtop-deploy/juju/hadoop-kafka/bundle-local.yaml
+++ b/bigtop-deploy/juju/hadoop-kafka/bundle-local.yaml
@@ -34,7 +34,7 @@ services:
gui-x: "1000"
gui-y: "400"
client:
- charm: "cs:xenial/hadoop-client-3"
+ charm: "cs:xenial/hadoop-client-8"
constraints: "mem=3G"
num_units: 1
annotations:
@@ -81,7 +81,7 @@ services:
to:
- "8"
ganglia:
- charm: "cs:~bigdata-dev/xenial/ganglia-5"
+ charm: "cs:xenial/ganglia-12"
num_units: 1
annotations:
gui-x: "0"
@@ -89,7 +89,7 @@ services:
to:
- "4"
ganglia-node:
- charm: "cs:~bigdata-dev/xenial/ganglia-node-7"
+ charm: "cs:xenial/ganglia-node-7"
annotations:
gui-x: "250"
gui-y: "400"
@@ -133,19 +133,28 @@ relations:
machines:
"0":
series: "xenial"
+ constraints: "mem=7G root-disk=32G"
"1":
series: "xenial"
+ constraints: "mem=7G root-disk=32G"
"2":
series: "xenial"
+ constraints: "mem=7G root-disk=32G"
"3":
series: "xenial"
+ constraints: "mem=7G root-disk=32G"
"4":
series: "xenial"
+ constraints: "mem=3G"
"5":
series: "xenial"
+ constraints: "mem=3G root-disk=32G"
"6":
series: "xenial"
+ constraints: "mem=3G root-disk=32G"
"7":
series: "xenial"
+ constraints: "mem=3G root-disk=32G"
"8":
series: "xenial"
+ constraints: "mem=3G"
diff --git a/bigtop-deploy/juju/hadoop-kafka/bundle.yaml b/bigtop-deploy/juju/hadoop-kafka/bundle.yaml
index 80aa8951..832d50d1 100644
--- a/bigtop-deploy/juju/hadoop-kafka/bundle.yaml
+++ b/bigtop-deploy/juju/hadoop-kafka/bundle.yaml
@@ -1,6 +1,6 @@
services:
namenode:
- charm: "cs:xenial/hadoop-namenode-12"
+ charm: "cs:xenial/hadoop-namenode-24"
constraints: "mem=7G root-disk=32G"
num_units: 1
annotations:
@@ -9,7 +9,7 @@ services:
to:
- "0"
resourcemanager:
- charm: "cs:xenial/hadoop-resourcemanager-13"
+ charm: "cs:xenial/hadoop-resourcemanager-26"
constraints: "mem=7G root-disk=32G"
num_units: 1
annotations:
@@ -18,7 +18,7 @@ services:
to:
- "0"
slave:
- charm: "cs:xenial/hadoop-slave-12"
+ charm: "cs:xenial/hadoop-slave-25"
constraints: "mem=7G root-disk=32G"
num_units: 3
annotations:
@@ -29,12 +29,12 @@ services:
- "2"
- "3"
plugin:
- charm: "cs:xenial/hadoop-plugin-12"
+ charm: "cs:xenial/hadoop-plugin-24"
annotations:
gui-x: "1000"
gui-y: "400"
client:
- charm: "cs:xenial/hadoop-client-3"
+ charm: "cs:xenial/hadoop-client-8"
constraints: "mem=3G"
num_units: 1
annotations:
@@ -52,7 +52,7 @@ services:
to:
- "4"
zookeeper:
- charm: "cs:xenial/zookeeper-16"
+ charm: "cs:xenial/zookeeper-30"
constraints: "mem=3G root-disk=32G"
num_units: 3
annotations:
@@ -63,7 +63,7 @@ services:
- "6"
- "7"
kafka:
- charm: "cs:xenial/kafka-11"
+ charm: "cs:xenial/kafka-27"
constraints: "mem=3G"
num_units: 1
annotations:
@@ -81,7 +81,7 @@ services:
to:
- "8"
ganglia:
- charm: "cs:~bigdata-dev/xenial/ganglia-5"
+ charm: "cs:xenial/ganglia-12"
num_units: 1
annotations:
gui-x: "0"
@@ -89,7 +89,7 @@ services:
to:
- "4"
ganglia-node:
- charm: "cs:~bigdata-dev/xenial/ganglia-node-7"
+ charm: "cs:xenial/ganglia-node-7"
annotations:
gui-x: "250"
gui-y: "400"
@@ -133,19 +133,28 @@ relations:
machines:
"0":
series: "xenial"
+ constraints: "mem=7G root-disk=32G"
"1":
series: "xenial"
+ constraints: "mem=7G root-disk=32G"
"2":
series: "xenial"
+ constraints: "mem=7G root-disk=32G"
"3":
series: "xenial"
+ constraints: "mem=7G root-disk=32G"
"4":
series: "xenial"
+ constraints: "mem=3G"
"5":
series: "xenial"
+ constraints: "mem=3G root-disk=32G"
"6":
series: "xenial"
+ constraints: "mem=3G root-disk=32G"
"7":
series: "xenial"
+ constraints: "mem=3G root-disk=32G"
"8":
series: "xenial"
+ constraints: "mem=3G"
diff --git a/bigtop-deploy/juju/hadoop-kafka/tests/01-bundle.py b/bigtop-deploy/juju/hadoop-kafka/tests/01-bundle.py
index fb113fc6..24f8bede 100755
--- a/bigtop-deploy/juju/hadoop-kafka/tests/01-bundle.py
+++ b/bigtop-deploy/juju/hadoop-kafka/tests/01-bundle.py
@@ -27,8 +27,6 @@ class TestBundle(unittest.TestCase):
@classmethod
def setUpClass(cls):
- # classmethod inheritance doesn't work quite right with
- # setUpClass / tearDownClass, so subclasses have to manually call this
cls.d = amulet.Deployment(series='xenial')
with open(cls.bundle_file) as f:
bun = f.read()
diff --git a/bigtop-deploy/juju/hadoop-kafka/tests/tests.yaml b/bigtop-deploy/juju/hadoop-kafka/tests/tests.yaml
index a3b7803f..b9517421 100644
--- a/bigtop-deploy/juju/hadoop-kafka/tests/tests.yaml
+++ b/bigtop-deploy/juju/hadoop-kafka/tests/tests.yaml
@@ -1,5 +1,5 @@
reset: false
-bundle_deploy: false
+deployment_timeout: 3600
sources:
- 'ppa:juju/stable'
packages:
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-12 18:44:22 +0000