Today I learned how to get JBoss AS (Wildfly) to not lose its mind during a database server failover event (as occurs in the cloud quite often).

The config is simple, but finding comprehensive documentation is a bit challenging since most current JBoss docs require a RedHat subscription. So you’re stuck piecing things together from multiple sites that contain tidbits of what’s needed.

Note: This is for a DBaaS scenario (think AWS RDS or Azure SQL Database), where the DB load balancing is done downstream of your app server. If you’re specifying multiple servers per connection, you’ll have to do some Googling of your own.

Otherwise, you’ve probably got a datasource (DS) connection defined in standalone.xml (or elsewhere depending on your deployment) that looks sort of like this:

datasource jndi-name="java:jboss/datasources/defaultDS" enabled="true" use-java-context="true" pool-name="defaultDS" use-ccm="true"
    connection-urljdbc:mysql://SQL_DB_SERVER_URL:3306/DB_NAME/connection-url
    drivermysql_version_derp_derp_derp.jar/driver
    security
        user-nameUSERNAME/user-name
        passwordPASSWORD/password
    /security
/datasource

Adding these options make JBoss handle failovers a bit better.

datasource jndi-name="java:jboss/datasources/defaultDS" enabled="true" use-java-context="true" pool-name="defaultDS" use-ccm="true"
    connection-urljdbc:mysql://SQL_DB_SERVER_URL:3306/DB_NAME?autoReconnect=true/connection-url
    drivermysql_version_derp_derp_derp.jar/driver
    security
        user-nameUSERNAME/user-name
        passwordPASSWORD/password
    /security
    validation
        check-valid-connection-sqlSELECT 1/check-valid-connection-sql
        background-validationtrue/background-validation
    /validation
    pool
        flush-strategyIdleConnections/flush-strategy
    /pool    
/datasource

Checkout lines 2 and 8-14.

On line 2 we’ve added the autoReconnect=true option to the connection-url. This does exactly what it says. If a database connection attempt fails, JBoss will attempt to re-establish the connection instead of sulking in a corner like it does by default. But it needs a way to know that it should reconnect…

On lines 8-11, we’ve added a connection validation. I believe some DS drivers handle this on their own, but the MySQL JDBC drivers I’ve tested appear not to. This seems to be the standard workaround from what I could find but does have the downside of issuing wasteful queries on the DB. The “background-validation” setting helps a little by issuing the validation checks in a separate thread.

This section should force JBoss to drop dead DB connections instead of letting them gum up the pipes.

Lines 12-14 help with the same problem. By default, JBoss is supposed to flush stale connections (that’s what the docs say at least), but this doesn’t seem to always happen in practice. Using IdleConnections should take care of any failed connections that aren’t getting flushed or EntirePool can be used if you want to be really aggressive.