Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The settings related to memory size don't work #630

Open
kamikyo opened this issue Sep 3, 2024 · 3 comments
Open

The settings related to memory size don't work #630

kamikyo opened this issue Sep 3, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@kamikyo
Copy link

kamikyo commented Sep 3, 2024

Describe the bug

I continuously write data to Garnet, primarily with three keys: two sorted sets and one string. Each time, I write 5000 entries, approximately 1 to 2 MB. Garnet's memory keeps increasing over time. As long as I keep writing data, the memory growth will not stop.

Steps to reproduce the bug

garnet server's code:

public static void Main(string[] args)
{
    try
    {
        var factory = LoggerFactory.Create(builder =>
        {
            builder.AddConsole(options =>
            {
                options.LogToStandardErrorThreshold = LogLevel.Trace;
            });
        });
        using var server = new GarnetServer(args, factory);
        server.Start();
        Thread.Sleep(Timeout.Infinite);
    }
    catch (Exception ex)
    {
        Console.WriteLine($"Unable to initialize server due to exception: {ex.Message}");
    }
}

the parameters I specified are:
--port 7010 --index "256m" --index-max-size "512m" --obj-index-max-size "512m" --memory "1g" --recover true --compaction-type "Scan" --checkpointdir "data" --storage-tier true

Expected behavior

I expect the memory to stabilize after reaching a certain limit.

Screenshots

No response

Release version

v1.0.15

IDE

Visual studio 2022 community

OS version

windows 10

Additional context

After each execution of the save command, memory usage increases by 200-300 MB.

@badrishc
Copy link
Contributor

badrishc commented Sep 3, 2024

Can you provide a sample client app as well? Try compaction type of Lookup.

If you are adding new entries in the sorted set, the total memory is expected to grow.

Also, it might be GC uncollected memory - try issuing the GC collection (FORCEGC) using the remote client command line.

@kamikyo
Copy link
Author

kamikyo commented Sep 4, 2024

@badrishc
The code for the garnet server is as described above. It is a console application(GarnetSample.csproj), and after publishing, it generates an executable file named GarnetSample.exe.
Server's start command:
GarnetSample.exe --port 7010 --index "256m" --index-max-size "512m" --obj-index-max-size "512m" --memory "1g" --recover true --compaction-type "Scan" --checkpointdir "data" --storage-tier true

Client project is DataCreator.csproj.It writes 5,000 records to the garnet server every 3 seconds.
The code repository is located at: https://github.com/kamikyo/garnetdemo

@badrishc
Copy link
Contributor

badrishc commented Sep 4, 2024

Does the problem persist if you use CompactionType of Lookup?

@darrenge darrenge added the bug Something isn't working label Sep 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants