DSS includes an audit trail that logs all actions performed by the users, with details about user id, timestamp, IP address, authentication method, …
You can view the latest audit events directly in the DSS UI: Administration > Security > Audit trail.
Note that this live view only includes the last 1000 events logged by DSS, and it is reset each time DSS is restarted. You should use log files or external systems for real auditing purposes.
The audit trail is logged in the
run/audit folder of the DSS data directory.
This folder is made of several log files, rotated automatically. Each file is rotated when it reaches 10 MB, and up to 10 history files are kept
In order to provide non-repudiation characteristics, it is critical that the audit log files are sent outside of the DSS machine, on a remote system that cannot be accessed and tampered with from the DSS machine.
The audit system in DSS sends data using the log4j library. Therefore, all log4j appenders can be used to get audit out of the DSS machine. Configuring log4j is done by editing the
resources/logging/dku-log4j.properties file. (See Logging in DSS for more information)
The following section includes examples of how to send audit to some external systems
Apache Kafka is a distributed message queue, which can be used to get audit logs out of DSS
To enable audit queries to Kafka:
- Add all jars from the Kafka distribution to the
lib/javafolder of your DSS datadir
- Create or edit the
resources/logging/dku-log4j.propertiesfile (See Logging in DSS for more information)
- Add the following lines
log4j.appender.AUDIT_KAFKA=org.apache.kafka.log4jappender.KafkaLog4jAppender log4j.appender.AUDIT_KAFKA.BrokerList=kafka1:9092,kafk2:9092,kafka:9093 log4j.appender.AUDIT_KAFKA.Topic=dku-dss-audit log4j.appender.AUDIT_KAFKA.layout=com.dataiku.dip.logging.JSONAuditLayout log4j.logger.dku.audit= INFO, AUDIT_KAFKA
- Replace dku-dss-audit by the name of your Kafka topic
- Replace BrokerList by the bootstrap list of your Kafka brokers
Your Kafka libraries must correspond to the version of your Kafka server