I've talked about Ansible a number of times, including mentioning it in my last post. I will take a second to mention that I am trying to build up a descriptive summary page about any topic I feel important enough to assign a category (or as Drupal calls them, Taxonomy term) to, but I have some cleanup to do, but if you are new to my page, it is a software automation platform which was written by the same guy who gave us Cobbler and Koan, and was acquired by Red Hat (he may have been employed by them at the time, and allowed him to release a open-source version while working on their dime... some companies have done that in the past, but not as often these days, sadly). But anyways, between other things I have had to do the past couple of days, I have been working on switching a ``MySQL'' server from the MariaDB 10.5 forked implementation to the official MySQL 8.4 install. And I ran into some issues.
Ansible has what they call "playbooks", which are best compared to a director in a movie or a conductor in a symphony, and perhaps a bit like a movie producer as well. They tell all the parts what to do, and sometimes, it can be a "self-directed/self-produced" production, where the playbook contains tasks which it itself does, perhaps with nobody else involved. Those are the simple ones. And they can be impromptu, where they are 100% self contained, or they can use variable files just like a play script, telling them what things to install, what versions, where, and such. And then, the really complicated parts, they get split out into things Ansible calls "roles" and "collections". Many times, all a playbook may do is say "I want roles X, Y and Z" to be done. Click the "Ansible" link at the beginning of the article, as well as the category below the article for more links. But just like playbooks, "roles" wrap together a whole bunch of modified playbooks, and can depend on other roles, or even on things in collections, which are just a bunch of collected roles. But all at the top sits a playbook, saying what machines to use, and either defining variables directly, or depending on the system-wide machine inventory to get variables based on the machine or what groups the machine belongs to, such as "Webservers" or "MySQL Servers". And a given machine can belong to multiple groups.
As I said, I ran into some issues. There always are bugs in any piece of software, it's just that many times, we don't see them. And yesterday, I did. The playbook I use for the overall provisioning of the machine where this site and others runs, and which also provisions the ``MySQL'' server involved needed updating. I had just said "Install mariadb", with something like this:
packages:
- some-package
- mariadb
- mariadb-server
- some-other-package
...And because MariaDB packages are a part of the regular set of packages for RHEL 9, that was all I needed. But, "MySQL" comes from Oracle, and Red Hat does not include the repository information for that. So I have been updating the playbook and roles, including a new one written to install the repo information for MySQL. And while doing so, I decided to try to get Ansible to re-create the users I had on the MariaDB server on MySQL, and got an error from the playbook and role stating that there was no database selected. So, I got into the whole "I want to upgrade this, which means I also have to upgrade this first" mess. And rather than being like a Python project or a PHP project, where tools like pip or composer know how to read a file and upgrade everything, it starts getting a bit like a bit like a brain surgeon operating on their own brain. But I asked, and after yesterday's pain, I see that Jeff Geerling, who has written so many roles I have downloaded to use in my playbooks, has talked about how to get Ansible to do just that. I just never looked at doing it that way... yet.
But yea, like pip or composer are for Python and PHP, ansible-galaxy is kinda like that. Only, so far, using that utility by hand has shown me that:
- Unlike
pipandcomposer,ansible-galaxydoes not have the ability to automatically (or perhaps I should say "auto-magically") look at some data about the role or collection and download the right versions for it, and continue the process until either every dependency is resolved, or it finds a conflict. - There really is no nice dependency versioning for roles, nor is there a way to manage the dependencies of a playbook, at least with the Community Edition of Ansible. Perhaps that is something which Red Hat's Ansible Automation Platform provides. I don't know, but I really should find out, especially since, as I discussed in my last post, I have a license to use it through my Red Hat Developer License.
- On top of those two, there is no easy way to say "Upgrade the roles I have downloaded..." or as "What roles do I have which are outdated". I literally have to look at things piece by piece and do this. I did find out that like the old days of Python, I could maintain a "requirements" file, in this case, in YAML format as opposed to a text file, and that helps. How much, I don't quite know yet, as I have only used it once. And, as I said in the previous item, there is no nice dependency versioning to help me avoid conflicts. Nor, unlike with
pip, is there a way to say "Please give me a list of the current roles in the format you understand for reinstalling things". For bothpipand its newer replacementuvin the Python community, they can do just that, whilecomposercannot do that for the PHP world... it just helps keep track of the new stuff you add, and if you somehow were doing things manually before, you have to recreate the file first. Interestingly enough,pipdid not keep the file updated automatically. Instead you had to saypip freezeand capture the output, but every time you did apip addto add a new dependency, you had to do that manually.uvon the other hand does that for you, which is why I am in the process of changing my Python workflow to use it instead, just like I went from therequirements.txtfile as output bypip freezeto using apyproject.tomlfile instead. But I digress. - Add to this, or perhaps because of all the above (and particularly #1), there is no way to indicate, other than in documentation what the dependencies are for a role package on Ansible Galaxy. Some developers do really good about saying what outside of the core of Ansible is needed as a dependency. Sometimes, that is not the case, especially if, for example it is only to do one specific thing, such as adding users to the MySQL server, which is where I hit one of my bugs yesterday. And the idea of each role containing its own
requirements.ymlfile... never seen that used, though it would be an improvement. - There is no way to have a role include a standardized playbook associated with it. But honestly, this is a nothing-burger, as because playbooks say what hosts to execute on, which can often be just a single machine or a specific group of machines, they tend to get customized, so I just include a sample in the documentation and in the role if I publish it.
Yes... while written in Python, Ansible is lacking features which would be so useful, and are thus on my wish-list. If only Ansible roles and collections were more like the Python code and packages which runs them, and it is something which I wish would be implemented, and not just for collections, but fully for roles as well.
Oh, and one more thing... at times, while I may only have a few tasks associated with a given playbook, I at times find myself wanting to copy a server to the machine I am managing, and playbooks really don't have a way to have files associated with them the way roles do. So, in those cases, I have just taken to using an "unofficial" and probably not best practice habit of putting them in a directory I created under /etc/ansible named files, and templates under templates, though by that point, I am starting to behave more like a role, and I frequently end up splitting the tasks out into a new role I create, and move those files into the directory structure of a role, with its own role co-dependencies, etc., and have the playbook just define the dependency on roles.
Well, that is pretty much it. I am currently working around a couple of apparent bugs or deficiency, like Jeff Geerling's geerlingguy.mysql role not installing the repo file for MySQL as provided by Oracle, or the ansible.mysql collection and its mysql_user module, which appears to have a issue dealing with newer versions of MySQL, or at least gets errors when trying to add a user stating that "A database is not selected". But, last night, before I headed to bed (I had an appointment around lunch time), I had reloaded my new MySQL server with the data from a backup of my old MariaDB server, tried tuleap, only to once again, complain. At least, this time, learning from and using the same technique I learned from figuring out the problem the last go-around, I know what is wrong. While the page at Oracle clearly uses the string "mysql84" in the name of the package which delivers the repository information to install MySQL 8.4 (tuleap wants only 8.0.x or 8.4.x), the file actually defaults to MySQL 9.7, which tuleap told me it was definitely not happy with. So now, add a couple of commands to my role for installing the MySQL repo to disable the 9.7 repositories and enable the 8.4 repositories, running a quick restore script I wrote to be able to go back to the MariaDB setup, and a quite simple ansible-playbook myhost.yml, and a manual restore of the database and users from backup (which I guess I could add to the role or playbook, but won't because that is an unusual task using an unusual backup), and I should have tuleap up, happy and running.