1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
|
<!DOCTYPE html>
<html>
<head>
<title>QMidiPlayer Help</title>
<link rel=stylesheet href='styles.css' type='text/css'>
</head>
<body>
<div id="panel">
<ul>
<li><a href="index.html">Overview</a></li>
<li><a href="mainwindow.html">Main Window</a></li>
<li><a href="channeldialog.html">Channels Dialog</a></li>
<li><a href="channeleditor.html">Channel Editor</a></li>
<li><a href="plistdialog.html">Playlist</a></li>
<li><a href="efxdialog.html">Effects</a></li>
<li><a href="optionsdialog.html">Settings</a></li>
<li><a class="active" href="visualization.html">Visualization</a></li>
<li><a href="cmdargs.html">Commandline arguments</a></li>
<li><a href="troubleshooting.html">Troubleshooting</a></li>
<li><a href="miscellaneous.html">Miscellaneous</a></li>
<li><a href="version.html">Version Info.</a></li>
<li><a href="license.html">License</a></li>
</ul>
</div>
<div id="content">
<h1>Visualization</h1>
<img src="../img/visualizationss.png"><br>
The default visualization comes as a plugin of QMidiPlayer. So before using it you
have to enable it first in the plugin manager.<br>
To use the visualization, click the Visualization button in the main window.<br>
The visualization plugin adds two new option tabs.
<h3>Controls</h3>
<ul>
<li>W: move camera forward</li>
<li>A: move camera left</li>
<li>S: move camera backward</li>
<li>D: move camera right</li>
<li>Q: move camera up</li>
<li>E: move camera down</li>
<li>Left arrow key: seek backward 1%</li>
<li>Right arrow key: seek forward 1%</li>
<li>Shift: seed 5% instead of 1%</li>
<li>Left mouse button: hold and drag to orient the camera</li>
</ul>
<h3>Options</h3>
The options listed here are applied after closing and reopening the visualization.
<ul>
<li>
Visualization-Appearance
<ul>
<li>Show Piano: Whether to show the virtual piano in the visualization scene. Currently drawing the piano is very expensive, so it is not recommended to enable this unless you have a beefy computer.</li>
<li>3D Notes: 2D notes are used when this is unchecked. Using 2D notes is less resource-hungry.</li>
<li>Arrange channels on a stair: If checked, virtual pianos will be arranged on a stair-like shape. This option has no effect if virtual piano is not shown.</li>
<li>Show channel labels: If checked, channel preset will be shown on the left side.</li>
<li>Show particles: Whether to draw particles. Very resource-intensive!</li>
<li>Horizontal Visualization: Uses a horizontal visualization style. Overrides everything above except 3D Notes.</li>
<li>2D Visualization: Tick this to use a simple 2D visualization instead. Much less resource-demanding.</li>
<li>Use spectrum instead of piano roll: Draw spectrum-like bars over the piano.</li>
<li>View distance: This option affects the maximum number of notes rendered on the screen. Only applies to the 3D visualization.</li>
<li>Note stretch: The length multiplier of notes.</li>
<li>Minimum note length: Avoid notes that are too short to be visible by adjusting this value.</li>
<li>Chequer board tint (AARRGGBB): change the color of the chequer board background.</li>
<li>Background Image: Use a background image instead of the default dull grey color.</li>
</ul>
</li>
<li>
Visualization-Video
<ul>
<li>Enable VSync: Enable vertical synchronization.</li>
<li>Save Viewport: Restore last camera configuration when the visualization is started.</li>
<li>Window Width/Height: Change the window size. If the size equals to your screen size, the visualization will start in fullscreen mode.</li>
<li>Target FPS: FPS limit of the visualization.</li>
<li>Supersampling: Supersample anti-aliasing for the 3D visualization scene. 1 means no SSAA.</li>
<li>Multisampling: Multisample anti-aliasing for the 3D visualization scene. 0 means no MSAA.</li>
<li>FOV: Field of view.</li>
<li>OSD Position: Change position of the on screen display, or simply disable it.</li>
<li>Font size: Change font size used by the visualization. Useful for HiDPI screens.</li>
</ul>
</li>
<li>
Key-only options<br>
Options listed here does not provide an entry in the option GUI. You have to add them to the configuration file manually if they don't exist in it.
<ul>
<li>px py pz rx ry rz (automatically created if save viewport is set to true): Doubles. Saves the last viewport. (px,py,pz): Position of the camera. (rx,ry,rz): Orientation of the camera.</li>
<li>chActiveColor<channel id 0..15> : Unsigned 32-bit integers. Colors of sounding notes.</li>
<li>chInactiveColor<channel id 0..15> : Unsigned 32-bit integers. Colors of normal notes.</li>
</ul>
</li>
</ul>
<h2>Visualization Renderer</h2>
The visualization plugin comes with a standalone application that generates high quality rendering of a midi file.
<h3>Command line usage</h3>
<pre style="font-variant-ligatures:none;">
Usage: ./qmpvisrender [options] file
Renderer a visualization of a midi file.
Options:
-h, --help Displays help on commandline options.
--help-all Displays help including Qt specific options.
-v, --version Displays version information.
-f, --output-file <filename> File name of the output file.
--receiver <command> Specify a program and its arguments to
process the rendered frames. Supports
parameter substitution. See documentation for
details.
-e, --receiver-execution <mode> Execution mode of the receiver command.
Valid options are 'one-shot' and 'per-frame'
-s, --show-window Do not hide the visualization window.
-c, --config <qmprc file> Load options from the configuration file.
-o, --option <key-value pair> Set option for the visualization module.
--list-options Show a list of recognized options.
Arguments:
file MIDI file to render
</pre>
<h3>Basic usage</h3>
You will most likely to load your QMidiPlayer configuration file so that
you can get the exact same results as viewed in QMidiPlayer.
You should also make sure ffmpeg is ready to use. Then you can run
<code>qmpvisrender -c <path-to-configuration-file> <MIDI file to render></code>
and wait for the results. The output file is named output.mp4 and will be found
in the current work directory.
<h3>Supplementary materials for commandline options</h3>
<p>
When specifying a value for an option under the "Visualization" category, the category part
in the key may be omitted, so "-o Visualization/wwidth=1920" would become "-o wwidth=1920".
</p>
<p>
The renderer performs parameter substitution before invoking the receiver command line.
</p>
<table>
<style>
td,th{
padding-top:0.5em;
padding-bottom:0.5em;
}
</style>
<tr><th>Placeholder</th><th>Substituted with</th><th>Remarks</th></tr>
<tr><td><code>%w</code></td><td>width of an output frame in pixels</td><td></td></tr>
<tr><td><code>%h</code></td><td>height of an output frame in pixels</td><td></td></tr>
<tr><td><code>%r</code></td><td>framerate which the output frames should be played at</td><td></td></tr>
<tr><td><code>%i</code></td><td>input format specification for ffmpeg</td><td>shorthand for "-f rawvideo -pixel_format rgba -video_size %wx%h -framerate %r -i pipe:"</td></tr>
<tr><td><code>%o</code></td><td>output file name specified by the --output-file option</td><td>If the receiver execution mode is "per-frame", "%f" in the output file name will also be replaced.</td></tr>
<tr><td><code>%f</code></td><td>a six-digit frame number</td><td>only works if the receiver execution mode is "per-frame".</td></tr>
</table>
<p>
To add a literal space in the receiver command, put a single backslash before the space.
To add a percent sign, either put two consecutive percent signs or use a backslash followed
by a percent sign.
</p>
<p>
The default receiver command is <code>ffmpeg %i -vf vflip -pix_fmt yuv420p -c:v libx264 -preset slow -crf 22 %o</code>.
In case the output quality doesn't satisfy your needs, consider lowering the number after '-crf' or selecting a even slower
profile ('slower' or 'veryslow').
</p>
<h3>How it works</h3>
<p>
Notice: you don't have to understand anything under this section to use this tool.
You may try out the examples below or simply adhere to the basic usage shown above even this
section makes zero sense to you.
</p>
<p>
The renderer feeds a sequence of images into one command or a series of commands that process
the images through a pipe. That's it!
</p>
<p>
The data is a stream of raw RGBA values, where each color takes one byte in every pixel. The frame
size is the same as used by the visualization module, and the frames should be played back at a fixed
frame rate. The pixel data starts from the bottom-left of a frame and follows row-major ordering (so
you may want to flip the frame when using the frame with some applications).
</p>
<p>
The renderer supports two modes of receiver program operation. If the mode is set to 'one-shot',
the receiver is only started once and gets all rendered frames. If it is set to 'per-frame',
the receiver will be started every time a frame is rendered, and will only get one frame to process.
</p>
<h3>Examples</h3>
<p>
If you wish to try these examples, make sure you have the required receiver program ready to run.
</p>
<ol>
<li>Render to a vp9 encoded webm video.</li>
<pre style="white-space:pre-wrap;">
qmpvisrender -c ~/.config/qmprc --output-file test.webm --receiver 'ffmpeg %i -vf vflip -c:v libvpx-vp9 -crf 30 -b:v 0 %o' <midi file>
</pre>
<li>Render to a h264 encoded mp4 video with hardware encoding acceleration using VA-API.</li>
<pre style="white-space:pre-wrap;">
qmpvisrender -c ~/.config/qmprc --output-file test.mp4 --receiver 'ffmpeg -vaapi_device /dev/dri/renderD128 %i -vf vflip,format=nv12,hwupload -c:v h264_vaapi -b:v 5M %o' <midi file>
</pre>
<li>Render the frames into png files using imagemagick.</li>
<pre style="white-space:pre-wrap;">
qmpvisrender -c ~/.config/qmprc --receiver-execution per-frame --receiver 'convert -size %wx%h -depth 8 RGBA:- -flip pngout/%f.png' <midi file>
</pre>
<li>Show the visualization window. Discard all rendered frames.</li>
<pre style="white-space:pre-wrap;">
qmpvisrender -c ~/.config/qmprc -s --receiver 'dd of=/dev/null bs=24M' <midi file>
</pre>
<li>Save the raw frame data.</li>
<pre style="white-space:pre-wrap;">
qmpvisrender -c ~/.config/qmprc --receiver-execution per-frame --receiver 'tee rawoutput/%f.raw' <midi file>
</pre>
</ol>
<h3>Useful stuff to refer to</h3>
<a target="_blank" href="https://www.ffmpeg.org/ffmpeg.html">ffmpeg documentation</a>
and <a target="_blank" href="https://www.ffmpeg.org/ffmpeg-codecs.html">ffmpeg codecs documentation</a>, if you want to tweak the ffmpeg encoder.
</div>
</body>
</html>
|
