Pull requests sent

After I finished my work on Salt, I wrote to the salt-users list, and they gave me some good piece of advice. Today I sent two pull requests to Salt.

The first one contains a fix to Salt's ps module [1]. It uses the psutil module, which breaks the compatibility with it's earlier versions. I added some wrapper functions, which are able hide the version differences. My syslog-ng module uses this ps module, and I had problems with it, so I had to fix it :)

The second pull request contains the prime cut of my work, the syslog-ng state and execution module is in this pull request. It can be auto merged, but the Jenkins build failed, but that's not my fault.

Tomorrow I will met with my mentor to talk about the further progress.

[1] https://github.com/saltstack/salt/pull/14032

[2] https://github.com/saltstack/salt/pull/14033

Work on Salt is in final state

I was out for a while, it's exam time in my university. Next to the studying I worked on my GSoC task but didn't write blog about it. Now, I have done with my exams so I can make up leeway.

I worked on a Salt state module and IMHO it's  development is reaching its final state. I'm done with config generation from YAML, you can find a howto here:

https://github.com/ihrwein/salt/wiki/Syslog-ng-state-HOWTO

There are a lot of examples I hope you will try them. I don't want to repeat myself, under the above link you can find information how to use the new syslog-ng state modul.

At first it was difficult to me to generate perfect syntax configurations. There are a lot of statement types (log, channel, source, options, etc.), some of them have names, some not. In a statement you can use already defined statements (I call them references) like source(s_local) or you can define them inline. The statements have options and the options have parameters. Some options have a type (like file("input.txt"), its type is file), but in some places you can omit them. For example you can write this:

Now you can see, you can't represent the file type as a dictionary. It should be a list, because its first item is just a string, there is no key for it. You must use different separator characters in different levels of the configuration tree. You can omit the colons in options, but in their parameters you can't. These are not very difficult problems but they are annoying until you get used to them. I built a recursive algorithm which tracks the level in the configuration and uses the appropriate separators, opening and enclosing characters.  You can find the source code here:

https://github.com/ihrwein/salt/blob/add-syslog-ng-module/salt/states/syslog_ng.py

Syslog-ng uses words yes and no in some places, so you have to use them in YAML also. PyYAML will load these variables as booleans if you don't enclose them in quotation marks:

http://salt.readthedocs.org/en/latest/topics/troubleshooting/yaml_idiosyncrasies.html#true-false-yes-no-on-off

If you have any questions or thoughts please leave a comment!

Collecting use cases

In my first post I wrote about myself and my attendance in GSoC 2014. I've already taken a look at Salt and wrote a proof of concept code. You can find it here:

https://github.com/ihrwein/salt/tree/add-syslog-ng-module

Recently I met with my mentor and got my first task: collect use cases.  The configuration management is a huge area and there is not just one software on the market: Salt, Chef, Puppet, Ansible (these are the most popular ones).

You can find below the summed up use cases that we managed to make up:

1. A lot of computers send their logs to a central log server,
2. syslog-ng listens on more than one interface and forwards the logs to a central log server,
3. you want to do some application layer filtering on log messages,
4. you may want to store application specific configuration files under a conf.d directory,
5. you might have template configuration files which can be used in more than one place (e.g. data centers),
6. grids: you got N new computers then you want them to apply the configuration of the "old" machines.

In the case of Salt, the requirements of a general modul are:

1. It should be some sort of high level stuff,
2. you may say: "I want a new destination, which is ...",
3. the users should have the opportunity to define the desired state ("I want a new destination to be created...")
4. it'd be nice to generate syslog-ng configuration from YAML files,
5. you might say: "Here is a file, take this as your config"
6. the infrastructure management is very significant. The syslog-ng configuration must be generatable (by using meta data).

I have some ideas about a Jinja templated solution, but I need more information about Salt to implement them. If you have any suggestions, feel free to share it with me!