ownCloud & Elasticsearch Integration

For a while I left Dropbox and other cloud storage solutions and decided to host my own file exchange service based on owncloud.org. I’m using it to exchange files with my partners and customers and keep a full control of the service from A to Z. A major advantage of ownCloud is its modular architecture which allows third party applications to be installed to extend its features. When I started to work with ownCloud, I wrote a first small application which adds a way to check the uploaded files against VirusTotal.

From my humble opinion, there is a point where ownCloud is lacking of good features: The way it manages events. By default, it is possible to send events to a remote Syslog server or in a flat file but the format of the generated events is really ugly. External application were developed to log events into a MySQL database but here again it was not enough convenient for me. Next to ownCloud, I’m also using ELK to manage my log files. It was clear that both solutions must be integrated and I wrote a small application which writes event directly into Elasticsearch. The idea and framework is based on SuperLog wrote by Bastien Ho.

ownCloud implements “hooks” that can be defined as:

A function whose name can be used by developers of plug-ins to ensure that additional code is executed at a precise place during the execution of other parts of ownCloud code. For example, when an ownCloud user is deleted, the ownCloud core hook post_deleteUser is executed.

An application can place a hook on post_deleteuser and automatically performs actions when a user is deleted. seLog supports the following hooks. For each of them, an event is sent to Elasticsearch with relevant information (source IP address, login, file, folder, etc) everytime the action is performed by a user or a desktop client.

• Users management
  • pre_login
  • post_login
  • logout
  • post_createUser
  • post_deleteUser
  • post_setPassword
  • post_createGroup
  • post_deleteGroup
  • post_addToGroup
  • post_removeFromGroup
• Filesystem
  • post_read
  • post_write
  • post_delete
  • post_rename
  • post_copy
  • file_put_contents
• Shared files
  • post_shared
  • post_update_permission
  • post_set_expiration_date
  • post_unshare
• Webdav
  • initialize
• Apps
  • post_enable
  • pre_disable

Before the esLog installation, the Elasticsearch PHP API must be deployed. Once done, you can setup the application like any other one. Extract the archive content into the /apps directory. To complete the installation, three manual steps must be performed:

1. Copy the “/vendor” directory created during the PHP API installation into a directory readable by Apache

2. Edit the file app/eslog/lib/log.php and add the following line at the top:

  require "/var/www/vendor/autoload.php"; # Change to your own location

3. To be able to log webdav operations, you must edit the remote.php file (in the root of ownCloud) and add the following line at the top:

  require_once 'apps/eslog/spy.php';

That’s it! Now enable the application via the admin panel and configure it. The following parameters can be defined:

• Elasticsearch host (default: 127.0.0.1:9200)
• Elasticsearch authentication mechanism (default: none)
• Elasticsearch user & password (default: blank)
• Elasticsearch index (default: owncloud)
• Elasticsearch type (default: owncloud)

Here is a dashboard example with data received from ownCloud:

The esLog application is available on my Github account or on the official ownCloud apps repository. Comments, suggestions are welcome and happy logging!