Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: voice recording message attachment #2311

Merged
merged 25 commits into from
Mar 7, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
447cb6a
feat: add audio recording message attachment widget
MartinCupela Feb 29, 2024
f4f7391
feat: use Largest-Triangle-Three-Buckets (LTTB) algorithm for voiceRe…
MartinCupela Mar 4, 2024
3f2a3cb
test: add tests for WaveProgressBar component
MartinCupela Mar 5, 2024
b79893b
feat: use HTMLAudioElement play API to reproduce the audio
MartinCupela Mar 5, 2024
44cc603
feat: support dragging progress indicator in WaveProgressBar
MartinCupela Mar 5, 2024
91e9a03
Merge branch 'master' into feat/audio-recording-message-attachment
MartinCupela Mar 5, 2024
aa7e0af
refactor: rename audio recording to voice recording
MartinCupela Mar 5, 2024
b942100
docs: document VoiceRecording component
MartinCupela Mar 5, 2024
284641d
feat: provide translations to VoiceRecording components
MartinCupela Mar 5, 2024
5614be8
refactor: remove dead code
MartinCupela Mar 5, 2024
a5c6f54
fix: fix typo in --str-chat__wave-progress-bar__amplitude-bar-height …
MartinCupela Mar 6, 2024
75a032f
docs: remove mentions about audio player controller from voice-record…
MartinCupela Mar 6, 2024
c3fce6d
fix: prevent memoizing VoiceRecording
MartinCupela Mar 6, 2024
aa64046
fix: use timeupdate HTMLMediaElement event to update audio progress s…
MartinCupela Mar 6, 2024
7dcd309
fix: fix typo divMode to divMod
MartinCupela Mar 6, 2024
42d569e
test: add missing snapshot file
MartinCupela Mar 6, 2024
2b2ae0d
test: update WaveProgressBar snapshot
MartinCupela Mar 6, 2024
d29559b
fix: remove increasePlaybackRate memoization
MartinCupela Mar 6, 2024
ccfd5b9
fix: remove progress state variable from useAudioController
MartinCupela Mar 6, 2024
92fd08a
fix: pass simple CSS variable to ProgressBar root style
MartinCupela Mar 6, 2024
7f41302
docs: update voice-recording.mdx title
MartinCupela Mar 7, 2024
66a37e2
docs: reformat voice-recording.mdx
MartinCupela Mar 7, 2024
0468a55
chore(deps): bump stream-chat-js & @stream-io/stream-chat-css
MartinCupela Mar 7, 2024
2b5b2a9
test: remove flaky Channel tests
MartinCupela Mar 7, 2024
2b8ffba
chore(deps): bump stream-chat peer dependency
MartinCupela Mar 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,251 @@
---
id: voice-recording
title: Voice recording attachment
MartinCupela marked this conversation as resolved.
Show resolved Hide resolved
---

import GHComponentLink from '../../../_docusaurus-components/GHComponentLink';
import ImageShowcase from '@site/src/components/ImageShowcase';

import VoiceRecordingRequestPayload from "../../../assets/voice-recording-response-payload.png";
import VoiceRecordingPlayer from "../../../assets/voice-recording-player.png";
import VoiceRecordingPlayerInProgress from "../../../assets/voice-recording-player-in-progress.png";
import VoiceRecordingPlayerStoppedRepro from "../../../assets/voice-recording-player-stopped-repro.png";
import VoiceRecordingPlayerFileSizeFallback from "../../../assets/voice-recording-player-file-size-fallback.png";
import VoiceRecordingPlayerFallbackTitle from "../../../assets/voice-recording-fallback-title.png";
import VoiceRecordingPlayerMissingWaveformData from "../../../assets/voice-recording-empty-waveform-data.png";

import QuotedVoiceRecording from "../../../assets/voice-recording-quoted.png";
import QuotedVoiceRecordingFileSizeFallback from "../../../assets/voice-recording-quoted-file-size-fallback.png";
import QuotedVoiceRecordingFallbackTitle from "../../../assets/voice-recording-quoted-fallback-title.png";

Audio attachments recorded directly from the chat UI are called voice recordings. The SDK provides a default implementation called <GHComponentLink text='VoiceRecording' path='/Attachment/VoiceRecording.tsx'/>. The default component renders or <GHComponentLink text='VoiceRecordingPlayer component' path='/Attachment/VoiceRecording.tsx'/> or <GHComponentLink text='QuotedVoiceRecording' path='/Attachment/VoiceRecording.tsx'/>.

The `VoiceRecordingPlayer` component is displayed in the message attachment list and is used to reproduce the audio.

<ImageShowcase
border
items={[
{
image: VoiceRecordingPlayer,
caption: <span>VoiceRecordingPlayer component</span>,
alt: 'Image of the VoiceRecordingPlayer component',
},
]}
/>

Whereas `QuotedVoiceRecording` is used to display basic information about the voice recording in quoted message reply.

<ImageShowcase
border
items={[
{
image: QuotedVoiceRecording,
caption: <span>QuotedVoiceRecording component</span>,
alt: 'Image of the QuotedVoiceRecording component',
},
]}
/>

## Attachment payload

The response payload for the voice recording attachment comes with the following properties:

<ImageShowcase
border
items={[
{
image: VoiceRecordingRequestPayload,
caption: <span>Voice recording payload</span>,
alt: 'Image of the voice recording payload',
},
]}
/>

These properties serve the following purpose:

| Property | Description |
|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
MartinCupela marked this conversation as resolved.
Show resolved Hide resolved
| **asset_url** | the URL where the voice recording is downloaded from |
| **duration** | the audio duration in seconds |
| **file_size** | the file size in bytes (displayed as fallback to duration if duration is not available) |
| **mime_type** | the file type that is later reflected in the icon displayed in the voice recording attachment widget |
| **title** | the audio title |
| **type** | the value will always be `"voiceRecording"` |
| **waveform_data** | the array of fractional number values between 0 and 1. These values represent the amplitudes later reflected in the <GHComponentLink text='WaveProgressBar' path='/Attachment/components/WaveProgressBar.tsx'/> |

## VoiceRecordingPlayer

### Navigation

By clicking in the space of waveform or by dragging the progress indicator a user can navigate to a specific place in the audio track. The progress indicator is placed at the point of the click or drag end and the preceding amplitude bars are highlighted to manifest the progress.

<ImageShowcase
border
items={[
{
image: VoiceRecordingPlayerStoppedRepro,
caption: <span>VoiceRecordingPlayer navigation</span>,
alt: 'Image of the VoiceRecordingPlayer stopped in the middle of the reproduction',
},
]}
/>

### Playback speed change

The playback speed can be changed by clicking the <GHComponentLink text='PlaybackRateButton' path='/Attachment/components/PlaybackRateButton.tsx'/>. The button is visible only during the audio reproduction. The rate is changed by repeatedly clicking the <GHComponentLink text='PlaybackRateButton' path='/Attachment/components/PlaybackRateButton.tsx'/>. Once the highest rate speed is achieved, the next click resets the speed to the initial value.

<ImageShowcase
border
items={[
{
image: VoiceRecordingPlayerInProgress,
caption: <span>VoiceRecordingPlayer playing the audio</span>,
alt: 'Image of the VoiceRecordingPlayer playing the audio',
},
]}
/>

## UI Fallbacks

### Missing duration

If the duration is not available in the [attachment object payload](./#attachment-payload), `VoiceRecordingPlayer` as well as `QuotedVoiceRecording` component will display the attachment size instead.

<ImageShowcase
border
items={[
{
image: VoiceRecordingPlayerFileSizeFallback,
caption: <span>VoiceRecordingPlayer displaying file size instead of audio duration</span>,
alt: 'Image of the VoiceRecordingPlayer displaying file size instead of audio duration',
},
{
image: QuotedVoiceRecordingFileSizeFallback,
caption: <span>QuotedVoiceRecording displaying file size instead of audio duration</span>,
alt: 'Image of the QuotedVoiceRecording displaying file size instead of audio duration',
},
]}
/>


### Missing title

If the voice recording does not come with title, a fallback title is provided.

<ImageShowcase
border
items={[
{
image: VoiceRecordingPlayerFallbackTitle,
caption: <span>VoiceRecordingPlayer displaying the fallback title</span>,
alt: 'Image of the VoiceRecordingPlayer displaying the fallback title',
},
{
image: QuotedVoiceRecordingFallbackTitle,
caption: <span>QuotedVoiceRecording displaying the fallback title</span>,
alt: 'Image of the QuotedVoiceRecording displaying the fallback title',
},
]}
/>

### Missing `waveform_data`

If the `waveform_data` is an empty array, then no progress bar is rendered.

<ImageShowcase
border
items={[
{
image: VoiceRecordingPlayerMissingWaveformData,
caption: <span>VoiceRecordingPlayer without progress bar</span>,
alt: 'Image of the VoiceRecordingPlayer missing progress bar',
},
]}
/>


## Audio player controller

The API to control the voice recording player is provided by <GHComponentLink text='useAudioController' path='/Attachment/hooks/useAudioController.tsx'/>. The `VoiceRecordingPlayer` makes use of the following memoized functions:

1. `togglePlay` - to start and stop playing the record
2. `seek` - to change audio progress
3. `increasePlaybackRate` - to change the playback rate in the round-robin manner

Besides the above actions, the controller maintains state in the following form:

1. `isPlaying` - boolean flag informing the UI, whether the reproduction is in flight
2. `progress` - numeric fractional value between 0 and 100 representing the progress of the audio reproduction
3. `playbackRate` - currently selected playback rate value
4. `secondsElapsed` - related to `progress`, but representing the amount of seconds elapsed since the audio beginning
MartinCupela marked this conversation as resolved.
Show resolved Hide resolved

## Default components customization

The pattern of customization applied to the default `VoiceRecording` component will be the same:

1. Create a custom voice recording component (e.g. `CustomVoiceRecording`). It will serve as a wrapper component that renders `VoiceRecordingPlayer` resp. `QuotedVoiceRecording`. Pass props to these components.

2. Create a custom attachment component (e.g. `CustomAttachment`), that will be again a wrapper around the SDK's `Attachment` component. Pass the custom voice recording component to the `Attachment` component via prop `VoiceRecording`.

### Provide custom list of playback speeds

You can override the default list of playback rates by overriding the <GHComponentLink text='VoiceRecording' path='/Attachment/VoiceRecording.tsx'/> component.

Example:

```typescript jsx
import {
Attachment,
AttachmentProps,
VoiceRecordingPlayer,
VoiceRecordingProps,
Channel,
QuotedVoiceRecording,
} from 'stream-chat-react';

import {ChannelInner} from './ChannelInner';

const CustomVoiceRecording = ({ attachment, isQuoted }: VoiceRecordingProps) =>
isQuoted ? (
<QuotedVoiceRecording attachment={attachment} />
) : (
<VoiceRecordingPlayer attachment={attachment} playbackRates={[2.0, 3.0]} />
);

const CustomAttachment = (props: AttachmentProps) => (
<Attachment {...props} VoiceRecording={CustomVoiceRecording}/>
);

const App = () => (
<Channel Attachment={CustomAttachment}>
<ChannelInner/>
</Channel>
);

export default App;
```

### Remove title

This could be solved by customizing the styles. You can stop displaying the recording title by tweaking the CSS:

```css
.str-chat__message-attachment__voice-recording-widget__title {
display: none;
}
```

### Customize the fallback title

If you do not like our fallback title, you can change it by changing the translation key `"Voice message"`.

### Other customizations

If you would like to perform the following customizations:

- change the progress bar
- change the file icon SVG

We recommend you to assemble your own components, that serve to display voice recording data and allow for reproduction. Then you can pass those components to the custom attachment component as described above.

The reason is, that `VoiceRecordingPlayer` and `QuotedVoiceRecording` are considerably small components. The inspiration can be taken from the default components implementations.
7 changes: 6 additions & 1 deletion docusaurus/sidebars-react.json
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,12 @@
"components/message-components/ui-components",
"components/utility-components/avatar",
"components/utility-components/base-image",
"components/message-components/attachment",
{
"Attachment": [
"components/message-components/attachment",
"components/message-components/attachment/voice-recording"
]
},
"components/message-components/reactions",
"components/utility-components/date_separator"
]
Expand Down
12 changes: 12 additions & 0 deletions src/components/Attachment/Attachment.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import {
isMediaAttachment,
isScrapedContent,
isUploadedImage,
isVoiceRecordingAttachment,
} from './utils';

import {
Expand All @@ -19,10 +20,12 @@ import {
ImageContainer,
MediaContainer,
UnsupportedAttachmentContainer,
VoiceRecordingContainer,
} from './AttachmentContainer';

import type { AttachmentActionsProps } from './AttachmentActions';
import type { AudioProps } from './Audio';
import type { VoiceRecordingProps } from './VoiceRecording';
import type { CardProps } from './Card';
import type { FileAttachmentProps } from './FileAttachment';
import type { GalleryProps, ImageProps } from '../Gallery';
Expand All @@ -37,6 +40,7 @@ const CONTAINER_MAP = {
file: FileContainer,
media: MediaContainer,
unsupported: UnsupportedAttachmentContainer,
voiceRecording: VoiceRecordingContainer,
} as const;

export const ATTACHMENT_GROUPS_ORDER = [
Expand All @@ -45,6 +49,7 @@ export const ATTACHMENT_GROUPS_ORDER = [
'image',
'media',
'audio',
'voiceRecording',
'file',
'unsupported',
] as const;
Expand All @@ -68,10 +73,14 @@ export type AttachmentProps<
Gallery?: React.ComponentType<GalleryProps<StreamChatGenerics>>;
/** Custom UI component for displaying an image type attachment, defaults to and accepts same props as: [Image](https://github.com/GetStream/stream-chat-react/blob/master/src/components/Gallery/Image.tsx) */
Image?: React.ComponentType<ImageProps>;
/** Optional flag to signal that an attachment is a displayed as a part of a quoted message */
isQuoted?: boolean;
/** Custom UI component for displaying a media type attachment, defaults to `ReactPlayer` from 'react-player' */
Media?: React.ComponentType<ReactPlayerProps>;
/** Custom UI component for displaying unsupported attachment types, defaults to NullComponent */
UnsupportedAttachment?: React.ComponentType<UnsupportedAttachmentProps>;
/** Custom UI component for displaying an audio recording attachment, defaults to and accepts same props as: [VoiceRecording](https://github.com/GetStream/stream-chat-react/blob/master/src/components/Attachment/VoiceRecording.tsx) */
VoiceRecording?: React.ComponentType<VoiceRecordingProps<StreamChatGenerics>>;
};

/**
Expand Down Expand Up @@ -135,6 +144,7 @@ const renderGroupedAttachments = <
image: [],
// eslint-disable-next-line sort-keys
gallery: [],
voiceRecording: [],
},
);

Expand Down Expand Up @@ -169,6 +179,8 @@ const getAttachmentType = <
return 'media';
} else if (isAudioAttachment(attachment)) {
return 'audio';
} else if (isVoiceRecordingAttachment(attachment)) {
return 'voiceRecording';
} else if (isFileAttachment(attachment)) {
return 'file';
}
Expand Down
15 changes: 15 additions & 0 deletions src/components/Attachment/AttachmentContainer.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import * as linkify from 'linkifyjs';

import { AttachmentActions as DefaultAttachmentActions } from './AttachmentActions';
import { Audio as DefaultAudio } from './Audio';
import { VoiceRecording as DefaultVoiceRecording } from './VoiceRecording';
import { Gallery as DefaultGallery, ImageComponent as DefaultImage } from '../Gallery';
import { Card as DefaultCard } from './Card';
import { FileAttachment as DefaultFile } from './FileAttachment';
Expand Down Expand Up @@ -237,6 +238,20 @@ export const AudioContainer = <
</AttachmentWithinContainer>
);

export const VoiceRecordingContainer = <
StreamChatGenerics extends DefaultStreamChatGenerics = DefaultStreamChatGenerics
>({
attachment,
VoiceRecording = DefaultVoiceRecording,
isQuoted,
}: RenderAttachmentProps<StreamChatGenerics>) => (
<AttachmentWithinContainer attachment={attachment} componentType='voiceRecording'>
<div className='str-chat__attachment'>
<VoiceRecording attachment={attachment} isQuoted={isQuoted} />
</div>
</AttachmentWithinContainer>
);

export const MediaContainer = <
StreamChatGenerics extends DefaultStreamChatGenerics = DefaultStreamChatGenerics
>(
Expand Down
Loading
Loading