Restructure the configuration of bentoml API server

[bojiang]

Is your feature request related to a problem? Please describe.
ref: #1476 (comment)

The current configuration structure for the API server:

api_server:
  port: 5000
  enable_microbatch: False
  run_with_ngrok: False
  enable_swagger: True
  enable_metrics: True
  enable_feedback: True
  max_request_size: 20971520
  workers: 1
  timeout: 60

marshal_server:
  port: Null  # default: api_server.port when enable_marshal=True

The batching server is seen as a standalone part, independent from the API server.
We can also see the batching layer as a consistent part of the bentoml API server.

So that bentoml could

• enable the marshal(batching) server by default, making the metrics endpoint non-blocking.
• pass requests by UNIX domain socket to achieve better performance.

Describe the solution you'd like

from @ssheng

api_server:
  marshal(or batching_layer):
    enable: true
    connector:
      type: TCP/UDS
      host: 127.0.0.1 # TCP only
      port: Null # TCP only. Default: auto-detect
      path: /path/to/the/uds # UDS only.
  port: 5000
  run_with_ngrok: False
  enable_swagger: True
  enable_metrics: True
  enable_feedback: True
  max_request_size: 20971520
  workers: 1
  timeout: 60

This structure reflects the idea above.

Describe alternatives you've considered

Additional context

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[parano]

This is done in 6d64f33