Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add auditd logging to platform for intrusion detection #314

Merged
merged 8 commits into from
Jul 29, 2024

Conversation

rcrichton
Copy link
Member

@rcrichton rcrichton commented Jul 17, 2024

Summary by CodeRabbit

  • New Features

    • Added auditd service support for enhanced security and compliance during server provisioning.
    • Introduced a new Grafana dashboard for monitoring security audit logs, featuring panels for data visualization and analysis.
    • Implemented structured log scraping for auditd logs, improving log management and analysis capabilities.
    • Enhanced Ansible documentation for improved clarity on server access and user authentication processes.
  • Bug Fixes

    • Resolved issues by replacing include with import_tasks for better task organization in playbooks.
  • Chores

    • Updated configuration files to support new logging and monitoring features.

Copy link
Contributor

coderabbitai bot commented Jul 17, 2024

Walkthrough

The recent changes enhance the server provisioning setup by integrating the auditd service for improved security monitoring. Ansible playbooks have been updated to install and configure auditd, while Grafana now features a dedicated dashboard for visualizing audit logs. Additionally, logging configurations in Promtail have been restructured to better handle both container and audit logs, providing a more organized approach to log management and analysis.

Changes

Files Change Summary
infrastructure/ansible/playbooks/...provision_servers.yml Added auditd to the provisioning playbook to enhance security and auditing capabilities.
infrastructure/ansible/roles/auditd/tasks/main.yml Introduced tasks for managing auditd, including installation, configuration, and service restart to ensure proper monitoring of system calls.
infrastructure/ansible/roles/docker/handlers/main.yml Changed task inclusion from include to import_tasks for better performance and organization in handling Docker-related tasks.
monitoring/docker-compose.yml Enhanced Grafana dashboard configuration to include an auditlogs.json file for monitoring security audit logs, improving observability and data analysis capabilities.
monitoring/grafana/dashboards/security/auditlogs.json Created a new Grafana dashboard focused on visualizing security audit logs with various panels and annotations for effective monitoring.
monitoring/promtail/promtail-config.yml Restructured log scraping configuration by adding a new job for auditd logs and enhancing the containers configuration for better log processing.
infrastructure/ansible/README.md Updated documentation to clarify server access, SSH key requirements, and firewall configurations for Docker swarm setups.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Ansible
    participant Auditd
    participant Grafana
    participant Promtail

    User->>Ansible: Trigger server provisioning
    Ansible->>Auditd: Install and configure
    Auditd-->>Ansible: Confirm installation
    Ansible->>Grafana: Update dashboard configuration
    Grafana-->>User: Provide audit log dashboard
    User->>Promtail: Scrape logs
    Promtail-->>User: Display processed logs
Loading

🐰 "In fields of code where rabbits play,
New logs and audits come to stay.
With Grafana's charts, we hop with glee,
Monitoring logs, as safe as can be!
Provisioning servers, oh what a sight,
In the warren of logs, we’ll hop through the night!" 🐇


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@rcrichton rcrichton marked this pull request as ready for review July 26, 2024 10:24
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 97ba293 and 18042fe.

Files selected for processing (6)
  • infrastructure/ansible/playbooks/provision_servers.yml (1 hunks)
  • infrastructure/ansible/roles/auditd/tasks/main.yml (1 hunks)
  • infrastructure/ansible/roles/docker/handlers/main.yml (1 hunks)
  • monitoring/docker-compose.yml (2 hunks)
  • monitoring/grafana/dashboards/security/auditlogs.json (1 hunks)
  • monitoring/promtail/promtail-config.yml (1 hunks)
Files skipped from review due to trivial changes (1)
  • infrastructure/ansible/roles/docker/handlers/main.yml
Additional context used
yamllint
infrastructure/ansible/roles/auditd/tasks/main.yml

[error] 1-1: wrong new line character: expected \n

(new-lines)

Gitleaks
monitoring/grafana/dashboards/security/auditlogs.json

162-162: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


201-201: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

Additional comments not posted (13)
infrastructure/ansible/playbooks/provision_servers.yml (1)

7-7: LGTM! Addition of auditd role is correct.

The syntax is correct and the role is correctly placed under the roles section.

infrastructure/ansible/roles/auditd/tasks/main.yml (4)

2-5: LGTM! Task to install auditd is correctly defined.

The task uses apt to install auditd and ensures it is in the latest state.


7-10: LGTM! Task to fetch auditd config is correctly defined.

The task fetches the audit.rules file from a URL and places it in the correct destination.


12-17: LGTM! Task to ensure name_format is set to HOSTNAME is correctly defined.

The task uses lineinfile to ensure name_format is set to HOSTNAME in the auditd.conf file.


19-22: LGTM! Task to restart auditd service is correctly defined.

The task uses ansible.builtin.service to restart the auditd service.

monitoring/promtail/promtail-config.yml (2)

12-46: LGTM! Job configuration for containers is correctly defined.

The job configuration includes pipeline stages for log processing, such as JSON parsing, regex extraction, timestamp formatting, and label assignment.


48-69: LGTM! Job configuration for auditd is correctly defined.

The job configuration includes pipeline stages for log processing, such as regex extraction and label assignment.

monitoring/grafana/dashboards/security/auditlogs.json (4)

1-23: Annotations section looks good.

The annotations section is well-formed and correctly configured for Grafana.


29-208: Panels section looks good.

The panels are well-configured and provide useful visualizations and information about audit logs.

Tools
Gitleaks

162-162: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


201-201: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


214-264: Templating section looks good.

The templating section is well-configured for querying and filtering logs.


267-277: Time and metadata sections look good.

The time, timepicker, and other metadata sections are correctly configured.

monitoring/docker-compose.yml (2)

233-237: New entry for auditlogs.json in configs section looks good.

The new entry for auditlogs.json is correctly configured with the file path and associated labels.


67-68: New entry for auditlogs.json in services section looks good.

The new entry for auditlogs.json is correctly configured with the target and source paths.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (4)
infrastructure/ansible/README.md (4)

21-21: Improve readability with punctuation and preposition correction.

Consider adding a comma after "remote servers" and replacing "on" with "to".

- To authenticate users and to allow them to have sudo access on the remote servers your ssh key will need to be added to the `sudoers` var in the _/inventories/{ENVIRONMENT}/group_vars/all.yml_.
+ To authenticate users and to allow them to have sudo access to the remote servers, your ssh key will need to be added to the `sudoers` var in the _/inventories/{ENVIRONMENT}/group_vars/all.yml_.
Tools
LanguageTool

[uncategorized] ~21-~21: The preposition ‘to’ seems more likely in this position.
Context: ...s and to allow them to have sudo access on the remote servers your ssh key will ne...

(AI_HYDRA_LEO_REPLACE_ON_TO)


[uncategorized] ~21-~21: Possible missing comma found.
Context: ... them to have sudo access on the remote servers your ssh key will need to be added to t...

(AI_HYDRA_LEO_MISSING_COMMA)


23-23: Improve readability with punctuation correction.

Consider adding a comma after "docker access".

- To authenticate users and to allow them to have docker access you need to add your ssh key to the  `docker_users` var in the _/inventories/{ENVIRONMENT}/group_vars/all.yml_.
+ To authenticate users and to allow them to have docker access, you need to add your ssh key to the  `docker_users` var in the _/inventories/{ENVIRONMENT}/group_vars/all.yml_.
Tools
LanguageTool

[uncategorized] ~23-~23: Possible missing comma found.
Context: ... users and to allow them to have docker access you need to add your ssh key to the `d...

(AI_HYDRA_LEO_MISSING_COMMA)


27-27: Improve readability and grammar.

Consider replacing "An" with "Any" and rephrasing to avoid repetition.

- An pre-authorised user will need to run the `provision_servers.yml` playbook the first time to add your ssh key to the servers.
+ Any pre-authorised user will need to run the `provision_servers.yml` playbook initially to add your ssh key to the servers.
Tools
LanguageTool

[style] ~27-~27: You have already used this phrasing in nearby sentences. Consider replacing it to add variety to your writing.
Context: ...edentials. An pre-authorised user will need to run the provision_servers.yml playboo...

(REP_NEED_TO_VB)


37-37: Correct verb agreement.

Consider replacing "allow" with "allows" to agree with the subject.

- which only allow that communication to happen on the particular subset specified by this property.
+ which only allows that communication to happen on the particular subset specified by this property.
Tools
LanguageTool

[uncategorized] ~37-~37: This verb does not appear to agree with the subject. Consider using a different form.
Context: ... firewall on each node (UFW) which only allow that communication to happen on the par...

(AI_EN_LECTOR_REPLACEMENT_VERB_AGREEMENT)

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 18042fe and 4d63774.

Files selected for processing (1)
  • infrastructure/ansible/README.md (2 hunks)
Additional context used
LanguageTool
infrastructure/ansible/README.md

[uncategorized] ~21-~21: The preposition ‘to’ seems more likely in this position.
Context: ...s and to allow them to have sudo access on the remote servers your ssh key will ne...

(AI_HYDRA_LEO_REPLACE_ON_TO)


[uncategorized] ~21-~21: Possible missing comma found.
Context: ... them to have sudo access on the remote servers your ssh key will need to be added to t...

(AI_HYDRA_LEO_MISSING_COMMA)


[uncategorized] ~23-~23: Possible missing comma found.
Context: ... users and to allow them to have docker access you need to add your ssh key to the `d...

(AI_HYDRA_LEO_MISSING_COMMA)


[uncategorized] ~26-~26: “an” (indefinite article before a vowel sound) seems less likely than “any”.
Context: ...nch of Jembi staff's user credentials. An pre-authorised user will need to run th...

(AI_HYDRA_LEO_CP_AN_ANY)


[style] ~27-~27: You have already used this phrasing in nearby sentences. Consider replacing it to add variety to your writing.
Context: ...edentials. An pre-authorised user will need to run the provision_servers.yml playboo...

(REP_NEED_TO_VB)


[uncategorized] ~37-~37: This verb does not appear to agree with the subject. Consider using a different form.
Context: ... firewall on each node (UFW) which only allow that communication to happen on the par...

(AI_EN_LECTOR_REPLACEMENT_VERB_AGREEMENT)

Additional comments not posted (2)
infrastructure/ansible/README.md (2)

15-15: Clarification improves documentation.

The change to "server's domain name/s that you created via terraform" enhances clarity and flexibility in server identification.


25-25: Security improvement.

The change emphasizes the importance of removing unauthorized users, which enhances security.

@drizzentic drizzentic merged commit 041010b into main Jul 29, 2024
3 checks passed
@drizzentic drizzentic deleted the auditd-logging branch July 29, 2024 06:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants