Introduction
Custom services in Apache Ambari can be packaged and installed in many ways. Ideally, they should all be packaged and installed in the same manner. This document describes how to package and install custom services using Extensions and Management Packs. Using this approach, the custom service definitions do not get inserted under the stack versions services directory. This keeps the stack clean and allows users to easily see which services were installed by which package (stack or extension).
Management Packs
A management pack is a mechanism for installing stacks, extensions and custom services. A management pack is packaged as a tar.gz file which expands as a directory that includes an mpack.json file and the stack, extension and custom service definitions that it defines.
Example Structure
myext-mpack1.0.0.0
├── mpack.json
└── <contents>
mpack.json Format
The mpacks.json file allows you to specify the name, version and description of the management pack along with the prerequisites for installing the management pack. For extension management packs, the only important prerequisite is the min_ambari_version. The most important part is the artifacts section. For the purpose here, the artifact type will always be “extension-definitions”. You can provide any name for the artifact and you can potentially change the source_dir if you wish to package your extensions under a different directory than “extensions”. For consistency, it is recommended that you use the default source_dir “extensions”.
{
"type" : "full-release",
"name" : "myextension-mpack",
"version": "1.0.0.0",
"description" : "MyExtension Management Pack",
"prerequisites": {
"min_ambari_version" : "2.4.0.0"
},
"artifacts": [
{
"name" : "myextension-extension-definitions",
"type" : "extension-definitions",
"source_dir": "extensions"
}
]
}
Extensions
An extension is a collection of one or more custom services which are packaged together. Much like stacks, each extension has a name which needs to be unique in the cluster. It also has a version folder to distinguish different releases of the extension which go in the resources/extensions folder with <name>/<version> sub-folders.
An extension version is similar to a stack version but it only includes the metainfo.xml and the services directory. This means that the alerts, kerberos, metrics, role command order and widgets files are not supported and should be included at the service level. In addition, the repositories, hooks, configurations, and upgrades directories are not supported although upgrade support can be added at the service level.
Extension Structure
MY_EXT
└── 1.0
├── metainfo.xml
└── services
├── SERVICEA
├── ...
Extension metainfo.xml Format:
The extension metainfo.xml is very simple, it just specifies the minimum stack versions which are supported.
<metainfo>
<prerequisites>
<min-stack-versions>
<stack>
<name>BIGTOP</name>
<version>1.0.*</version>
</stack>
</min-stack-versions>
</prerequisites>
</metainfo>
Extension Inheritance
Extension versions can extend other Extension versions in order to share command scripts and configurations. This reduces duplication of code across Extensions with the following:
- add new Services in the child Extension version (not in the parent Extension version)
- override command scripts of the parent Services
- override configurations of the parent Services
For example, MyExtension 2.0 could extend MyExtension 1.0 so only the changes applicable to the MyExtension 2.0 extension are present in that Extension definition. This extension is defined in the metainfo.xml for MyExtension 2.0:
<metainfo> <extends>1.0</extends> |
|---|
Extension Management Packs Structure
myext-mpack1.0.0.0
├── mpack.json
└── extensions
└── MY_EXT
└── 1.0
├── metainfo.xml
└── services
└── SERVICEA
└── 2.0
├── metainfo.xml
└── services
├── SERVICEA
└── …
Installing Management Packs
In order to install an extension management pack, you run the following command with or without the “-v” option:
ambari-server install-mpack --mpack=/dir/to/myext-mpack-1.0.0.0.tar.gz -v
This will check to see if the management pack's prerequisites are met (min_ambari_version). In addition it will check to see if there are any errors in the management pack format. Assuming everything is correct, the management pack will be extracted in:
/var/lib/ambariserver/resources/mpacks.
It will then create symlinks from /var/lib/ambari-server/resources/extensions for each extension version in /var/lib/ambari-server/resources/mpacks/<mpack dir>/extensions.
Extension Directory | Target Management Pack Symlink |
resources/extensions/MY_EXT/1.0 | resources/mpacks/myext-mpack1.0.0.0/extensions/MY_EXT/1.0 |
resources/extensions/MY_EXT/2.0 | resources/mpacks/myext-mpack1.0.0.0/extensions/MY_EXT/2.0 |
Verifying the Extension Installation
Once you have installed the extension management pack, you can restart ambari-server.
ambari-server restart
After ambari-server has been restarted, you will see in the ambari DB your extension listed in the extension table:
ambari=> select * from extension;
extension_id | extension_name | extension_version
--------------+----------------+-------------------
1 | EXT | 1.0
(1 row)
You can also query for extensions by calling REST APIs.
curl -u admin:admin -H 'X-Requested-By:ambari' -X GET 'http://<server>:<port>/api/v1/extensions'
{
"href" : "http://<server>:<port>/api/v1/extensions",
"items" : [{
"href" : "http://<server>:<port>/api/v1/extensions/EXT",
"Extensions" : {
"extension_name" : "EXT"
}
}]
}
curl -u admin:admin -H 'X-Requested-By:ambari' -X GET 'http://<server>:<port>/api/v1/extensions/EXT'
{
"href" : "http://<server>:<port>/api/v1/extensions/EXT",
"Extensions" : {
"extension_name" : "EXT"
},
"versions" : [{
"href" : "http://<server>:<port>/api/v1/extensions/EXT/versions/1.0",
"Versions" : {
"extension_name" : "EXT",
"extension_version" : "1.0"
}
}]
}
curl -u admin:admin -H 'X-Requested-By:ambari' -X GET 'http://<server>:<port>/api/v1/extensions/EXT/versions/1.0'
{
"href" : "http://<server>:<port>/api/v1/extensions/EXT/versions/1.0",
"Versions" : {
"extension-errors" : [ ],
"extension_name" : "EXT",
"extension_version" : "1.0",
"parent_extension_version" : null,
"valid" : true
}
}
Linking Extensions to the Stack
Once you have verified that Ambari knows about your extension, the next step is linking the extension version to the current stack version. Linking adds the extension version's services to the list of stack version services. This allows you to install the extension services on the cluster. Linking an extension version to a stack version, will first verify whether the extension supports the given stack version. This is determined by the stack versions listed in the extension version's metainfo.xml.
The following REST API call, will link an extension version to a stack version. In this example it is linking EXT/1.0 with the BIGTOP/1.0 stack version.
curl -u admin:admin -H 'X-Requested-By: ambari' -X POST -d '{"ExtensionLink": {"stack_name": "BIGTOP", "stack_version": "1.0", "extension_name": "EXT", "extension_version": "1.0"}}' http://<server>:<port>/api/v1/links/
You can examine links (or extension links) either in the Ambari DB or with REST API calls.
ambari=> select * from extensionlink;
link_id | stack_id | extension_id
---------+----------+--------------
1 | 2 | 1
(1 row)
curl -u admin:admin -H 'X-Requested-By:ambari' -X GET 'http://<server>:<port>/api/v1/links'
{
"href" : "http://<server>:<port>/api/v1/links",
"items" : [{
"href" : "http://<server>:<port>/api/v1/links/1",
"ExtensionLink" : {
"extension_name" : "EXT",
"extension_version" : "1.0",
"link_id" : 1,
"stack_name" : "BIGTOP",
"stack_version" : "1.0"
}
}]
}