Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Memory consumption too high / Memory Leak? #4628

Closed
dragotin opened this issue Sep 21, 2022 · 3 comments · Fixed by #4660
Closed

Memory consumption too high / Memory Leak? #4628

dragotin opened this issue Sep 21, 2022 · 3 comments · Fixed by #4660
Assignees
@rhafer
Labels
Priority:p2-high Escalation, on top of current planning, release blocker Type:Bug
Milestone
2.0.0 General Ava...

Comments

@dragotin
Copy link
Contributor

Describe the bug

Running oCIS on a Raspberry Pi with 1 GB of memory does not work any more. oCIS starts up properly, but gets killed after the memory has grown to the limit of nearly 1GB.

It is not relevant if it is a Raspberry Pi or another system, it needs to be known and clear where the increased memory demand comes from or if it is a memory leak.

Steps to reproduce

Just start oCIS on a Raspberry Pi.
Test System: Raspberry Pi 3B with 1GB of RAM, running Raspbian Lite.

Expected behavior

oCIS should run on a Raspberry Pi with 1GB of memory properly.
@micbar micbar added this to the 2.0.0 General Availability milestone Sep 21, 2022
@micbar micbar added Priority:p2-high Escalation, on top of current planning, release blocker GA-Blocker labels Sep 21, 2022
@micbar
Copy link
Contributor

micbar commented Sep 21, 2022

We need to find out:

  • Do we have a memory leak?
  • Is the memory footprint in general too high?
  • Is the problem located in bleve or in the ocis implementation?

@rhafer rhafer self-assigned this Sep 23, 2022
@rhafer
Copy link
Contributor

rhafer commented Sep 23, 2022

profile001

This issue seems to be cause by some overly large caches that a configured in the reva auth provider. They alll hardcoded to 1 Million cached items and upon startup the cache implemenation does this: c.items = make(map[interface{}]*lfuItem, c.size) which causes a huge allocation. 💥

@rhafer
Copy link
Contributor

rhafer commented Sep 23, 2022

After lowering the cache to 10000 items (which is still quite large for a user cache IMO) looks a lot better (down to 60MB, from more than 1GB after startup):

profile002

rhafer added a commit to rhafer/reva that referenced this issue Sep 23, 2022
@rhafer
Reduce default cache sizes for smaller memory footprint
4e0e4d0 
We reduced the default cachesizes of the auth interceptors and the share
cache. The default of 1 Million cache entries was way too high and caused
a high memory usage upon startup. Config options to set custom cache size
where added.

https://github.com/cs3org/reva/pull/3xxx
owncloud/ocis#4628
@rhafer rhafer mentioned this issue Sep 23, 2022
Merged
rhafer added a commit to rhafer/reva that referenced this issue Sep 23, 2022
@rhafer
Reduce default cache sizes for smaller memory footprint
fa835be 
We reduced the default cachesizes of the auth interceptors and the share
cache. The default of 1 Million cache entries was way too high and caused
a high memory usage upon startup. Config options to set custom cache size
where added.

cs3org#3267
owncloud/ocis#4628
rhafer added a commit to rhafer/reva that referenced this issue Sep 23, 2022
@rhafer
Reduce default cache sizes for smaller memory footprint
f9ed40f 
We reduced the default cachesizes of the auth interceptors and the share
cache. The default of 1 Million cache entries was way too high and caused
a high memory usage upon startup. Config options to set custom cache size
where added.

owncloud/ocis#4628
butonic pushed a commit to cs3org/reva that referenced this issue Sep 26, 2022
@rhafer
Reduce default cache sizes for smaller memory footprint (#3267)
ebcfaf9 
We reduced the default cachesizes of the auth interceptors and the share
cache. The default of 1 Million cache entries was way too high and caused
a high memory usage upon startup. Config options to set custom cache size
where added.

owncloud/ocis#4628
rhafer added a commit to rhafer/ocis that referenced this issue Sep 26, 2022
@rhafer
Bump reva to latest edge (for owncloud#4628)
40fc8c8 
Fixes: owncloud#4628
@rhafer rhafer mentioned this issue Sep 26, 2022
Merged
rhafer added a commit to rhafer/ocis that referenced this issue Sep 26, 2022
@rhafer
Bump reva to latest edge (for owncloud#4628)
800bd65 
Fixes: owncloud#4628
rhafer added a commit to rhafer/ocis that referenced this issue Sep 26, 2022
@rhafer
Bump reva to latest edge (for owncloud#4628)
5a46ac5 
Fixes: owncloud#4628
rhafer added a commit to rhafer/ocis that referenced this issue Sep 26, 2022
@rhafer
Bump reva to latest edge (for owncloud#4628)
34cee3e 
Fixes: owncloud#4628
rhafer added a commit to rhafer/ocis that referenced this issue Sep 26, 2022
@rhafer
Bump reva to latest edge
08c09bc 
Fixes: owncloud#4628, owncloud#4657
butonic pushed a commit that referenced this issue Sep 27, 2022
@rhafer
Bump reva to latest edge (#4660)
918bc85 
Fixes: #4628, #4657
ownclouders pushed a commit that referenced this issue Sep 27, 2022
@rhafer
commit 918bc85
a07db2c 
Author: Ralf Haferkamp <rhaferkamp@owncloud.com>
Date:   Tue Sep 27 10:36:42 2022 +0200

    Bump reva to latest edge (#4660)

    Fixes: #4628, #4657
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rhafer rhafer

Labels
Priority:p2-high Escalation, on top of current planning, release blocker Type:Bug
Projects
None yet
Milestone
2.0.0 General Availability
Development

Successfully merging a pull request may close this issue.

[full-ci] Bump reva to latest edge (for #4628) rhafer/ocis
3 participants
@rhafer @dragotin @micbar