Keeping your VMs from going read-only when encountering a ping-timeout in GlusterFS

'GlusterFS communicates over TCP. This allows for stateful handling of file descriptors and locks. If, however, a server fails completely, kernel panic, power loss, some idiot with a reset button... the client will wait for ping-timeout (42 by the defaults) seconds before abandoning that TCP connection. This is important because re-establishing FDs and locks can be a very expensive operation. As glusterbot says in #gluster:' [Read more...]

How To Check (And Fix) Indexes For Elasticsearch/Logstash

I had a computer with some bad ram that created a corrupt index in Elasticsearch. After trying all weekend and half of Monday to figure out how to get Elasticsearch running again with no response on the IRC channel, I eventually figured it out with the help of some obscure email references to references. [Read more...]

Announcing the development of my Puppet module for dynamic GlusterFS volumes

'A couple times a year someone comes in to #gluster and thinks it would be awesome to ave a puppet module that you could define servers and the pupet module would automatically manage adding those servers as peers and dynamically changing volumes to utilize the new assets. They get inspired and start writing manifests... then they realize they actually have to get something completed and abandon that pie-in-the-sky philosophy and just use the very convenient cli.' [Read more...]

Fedora 19 with legacy GlusterFS 3.3

I went into this expecting problems. Fedora 19 ships with GlusterFS 3.4.0 beta and I'm using GlusterFS 3.3 in production. I expected that I would have to downgrade my Fedora packages so I could use my volumes. I expected problems. [Read more...]

Setting up Dogtag PKI CA Certificate Manager

Any service I can set up redundantly I, obviously, do. A couple years ago I set up redundant ldap servers. Last year my certificated expired and I generated new ones, not paying any attention to the serial id. Unfortunately, I generated two certificates with duplicate serials. sssd recognized that and failed to authenticate users via ldaps. My company's security requirements are relatively lax so I've just been authenticating over insecure ldap. That just annoys the hell out of me, though, and I've been wanting to fix it - but fix it the right way. Since I can't buy a CA from one of the big boys, I wanted to run my own CA server.  [Read more...]

PHP playing fast and loose with your data integrity

'Had a potential GlusterFS user state that the filesystem incorrectly reported that a write succeeded even though all the servers were powered off. Since this sounded rather impossible, I asked for details and duplicated the problem. This is the php code:' [Read more...]