How to serve big files from an SD Card?

[JRoseCPMax]

I'm currently trying to download ~20Mb Files from an ESP32 with an SD Card. I tried the following approach with the ChunkedResponse example:

#include <Arduino.h>
...
#include "FS.h"
#include "SD.h"
...
#include "WiFi.h"
#include "ESPAsyncWebServer.h"
#include "WebSocketsServer.h"

AsyncWebServer server(80);
...

size_t load_data(File f, uint8_t *buffer, size_t maxLen, size_t index){
  if (f.available()){
    // return f.read(buffer, maxLen);     // WDT
    return f.read();                      // extremely slow (6KB/s)
  } else {
    return 0;
  }
}

void onPageNotFound(AsyncWebServerRequest *request){
  IPAddress ip = request -> client() -> remoteIP();
  String requested_file = request->url();
  Serial.println("["+ip.toString() +"] GET Request of " + requested_file);

  if (requested_file.startsWith(folder) & SD.open(requested_file, FILE_READ)){
    File f = SD.open(requested_file, FILE_READ);
    AsyncWebServerResponse *response = request->beginChunkedResponse("text/csv", [f](uint8_t *buffer, size_t maxLen, size_t, index) -> size_t {
      return load_data(f, buffer, maxLen, index);
    });
    response->addHeader("Server","ESP Async Web Server");
    request->send(response);
    // f.close();                        // <-- don't know where to put this, it interrupts the transmission at this point
  } else {
    request->send_P(404, "text/html", "Not Found");
  }
}

void setup(){
...
 WiFi.softAP(ssid, password);
...
 server.on("/", HTTP_GET, onIndexRequest);
 server.onNotFound(onPageNotFound);
 server.begin();
}
...

When using f.read() in load_data it works but only extremly slow, i suspect, it transmits one char at a time?
However, when I use f.read(buffer, maxLen) and start the download the WDT gets triggered.
with request->send(SD, requested_file, "text/csv"); the header (Filesize) gets transmitted correctly, but the WDT also triggers.

E (54572) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (54572) task_wdt:  - async_tcp (CPU 0/1)
E (54572) task_wdt: Tasks currently running:
E (54572) task_wdt: CPU 0: IDLE0
E (54572) task_wdt: CPU 1: loopTask
E (54572) task_wdt: Aborting.
abort() was called at PC 0x400e9e17 on core 0

What is the most elegant way to serve a bigger file from SD?

EDIT: Now using the workaround of a second Server instance (WebServer) on a diff. port. ESP32 handles just fine.

[jendakol]

See #770

[stale]

[STALE_SET] This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[stale]

[STALE_DEL] This stale issue has been automatically closed. Thank you for your contributions.

[simkard69]

Did you managed to find a solution to serve big files from SD card without having the ESP32 crashing/restarting due to watchdog being triggered ?

Thanks

[JRoseCPMax]

No, unfortunately not. Restructured my programm to not use this library since it crashed my programm when losing signal during transmission. I now use the standard WebServer.h, which works very well with FreeRTOS and is very stable.

[simkard69]

Wow ok, so if WiFi signal is lost during transmission, all the code goes to crash too ?

... What a pain :(

[JRoseCPMax]

It was in my case, best you try it yourself. Maybe it was a weird synergy between my RTOS Tasks. I use the wifi core for a few other tasks too.

[AcuarioCat]

This works for me. The failure in the OP code is not sending a value for maxLen to read from the file. I use 2K and haven't tried anything bigger.

size_t load_data(File f, uint8_t* buffer, size_t maxLen, size_t index) {
if (f.available()) {
return f.read(buffer, maxLen);
}
else {
return 0;
}
}

In the webserver code:

server.on("/readFile", HTTP_GET, [](AsyncWebServerRequest* request) {
if (request->args() > 0)
{
if (request->hasParam("f")) {
File f = SD.open(file, FILE_READ);
AsyncWebServerResponse* response = request->beginChunkedResponse("text/plain", [f](uint8_t* buffer, size_t maxLen, size_t index) -> size_t {
return load_data(f, buffer, 2048, index);
});
request->send(response);
}
}
else
request->send(200);
});