Difference between revisions of "ETTV:Server Configuration"

From WolfWiki
Jump to: navigation, search
(copied from http://ettv.anime.net/documentation.html + merge release forum posts)
 
 
(16 intermediate revisions by 5 users not shown)
Line 1: Line 1:
 +
<graphviz>
 +
digraph ettv {
 +
master -> slave1
 +
master -> slave2
 +
slave1 -> "client1.."
 +
slave1 -> "..client50"
 +
slave2 -> "client51.."
 +
slave2 -> "..client100"
 +
slave1 -> "chained_slave1"
 +
chained_slave1 -> "client101"
 +
chained_slave1 -> "...client150"
 +
}
 +
</graphviz>
 +
 
==Installation - "Master" (gameserver)==
 
==Installation - "Master" (gameserver)==
  
Copy ettv.x86 to the same directory your etded.x86 is located.
+
Copy ettv.(x86|exe) to the same directory your etded.(x86|exe) is located.
Start your gameserver as normal, but run ettv.x86 instead of etded.x86.
+
Start your gameserver as normal, but run ettv.(x86|exe) instead of etded.(x86|exe).
  
 
===Set some cvars===
 
===Set some cvars===
Line 12: Line 26:
 
:Password for ettv slaves.
 
:Password for ettv slaves.
 
:Must be set, or no slaves will be able to connect!
 
:Must be set, or no slaves will be able to connect!
'''b_ettv_flags''' (default 1)
+
'''b_ettv_flags''' (default 3)
 
:1 = prevent slaves from being callvote kicked
 
:1 = prevent slaves from being callvote kicked
 
:2 = grant shoutcaster status to slaves
 
:2 = grant shoutcaster status to slaves
Line 22: Line 36:
 
==Installation - "Slave" (broadcast server)==
 
==Installation - "Slave" (broadcast server)==
  
Copy ettv.x86 to the same directory your etded.x86 is located.
+
Copy ettv.(x86|exe) to the same directory your etded.(x86|exe) is located.
Copy tvgame.mp.i386.so to the same directory as your etpro/qagame.mp.i386.so is located.
+
Copy tvgame.mp.i386.(so|dll) to the same directory as your etpro/qagame.mp.i386.(so|dll) is located.
  
 
===Set some cvars===
 
===Set some cvars===
Line 60: Line 74:
  
 
===Connect the slave===
 
===Connect the slave===
Now start your slave server by running ettv.x86
+
Now start your slave server by running ettv.(x86|exe)
  
 
Tell the slave server to connect up to the master server:
 
Tell the slave server to connect up to the master server:
Line 75: Line 89:
  
 
===Slave Chaining===
 
===Slave Chaining===
;something cool goes here
+
 
 +
==== What is chaining and why chain slaves? ====
 +
 
 +
ETTV allows chaining slaves together. This very neat feature lets you connect one ETTV slave to another ETTV slave that is broadcasting a match instead of directly to the game server hosting the match. Apart from the "w00t this is very cool" factor, why is this feature so important for admins? Basicly, working in a chained model allows to broadcast a match to a large number of spectators without too much hastle and too many risks for the master server.
 +
 
 +
Chaining was developped for a simple reason: if you take into account that one ETTV slave connects at an '''unlimited''' rate to a master, you really don't want to connect 15 or 20 slaves to a game server to broadcast let's say the ''Atlantic Battle''. Chances are you will bring down the master game server and thus the game. Chaining slaves firmly alleviates the load on the master game server. 2 or 4 of these ''Hubs'' can do the job of delaying the signal and feeding it to other slaves.
 +
 
 +
Other benefits might include better network performance. If you can choose to either connect your slave to the game server where your connection is not optimal or to a nearby slave who is on a better connection to the master, it might help the quality of your broadcast. Plus you don't need to record anything when connecting to a delayed slave.
 +
 
 +
==== Setting up your slave to allow connections from other slaves ====
 +
 
 +
This part is actually pretty easy. You need to set up your slave as if you were setting up a master game server. This means that, next to the normal slave parameters, you have to set up the following:
 +
 
 +
* '''b_ettv_flags''': set this to at least 1
 +
* '''ettv_sv_maxslaves''': set this to at least 1. Depending on how powerful your machine is you can set this number higher. Don't set it too high though, a slave still connects at an unlimited rate
 +
* '''ettv_password''': set to something else than "". This password is needed by the slaves connecting to your slave (together with g_password of course)
 +
 
 +
Other settings such as ettv_netblast, ettv_tv_iptos etc... can be set as well. You can even use autorecord if you wish, or just add an extra delay.
 +
 
 +
==== Pitfalls ====
 +
 
 +
There are however a few risks associated with slave chaining. One of the biggest issues is '''PK3 dependancies'''.
 +
 
 +
As you know, a slave needs to have at least all the PK3s of the server it connects too. But a slave can have more PK3s installed, as long as it has the ones on the server's pure list. This is what complicates things! The further you go down the "chain", the more the chances exist of PK3 pollution, which makes the job of the last slave very hard. And the viewers connecting to that slave will not be very happy either.
 +
 
 +
So for those of you venturing into chaining slaves here's a rule of thumb: '''always keep the installed PK3s to the bare minimum'''! Do not add stuff you do not need. If you want to know what PK3s you need and not need, connect your slave to the master or the hub, see which PK3s are on the pure list and rename or remove the PK3s that are not on the list. You will make your viewers and other admins connecting to your slave very happy.
 +
 
 +
Another risk could be '''chained lag'''. If one of the slaves that is higher in the chain starts lagging, the consequenses for the slaves depending on it will be seen immedeatily. So if you offer to be one of the higher slaves in the chain, make sure your machine can handle it! An ETTV chain is much like the food chain ;-).
 +
 
 +
==== Admin contributions ====
 +
 
 +
If by now you feel like getting your hands dirty in ETTV chaining madness and you want to see some example of hubs, slaves, broadcasters and scripts to get it all working, you can already start by checking out [[User:Deej|deej's page]].
  
 
==ETTV Demos==
 
==ETTV Demos==
 +
ETTV can record and playback '''ETTV demos'''. ETTV demos have the extension .tv_84 (or .tv_83 for older versions) and are completely different from the .dm_84 demos recorded on clients.
 +
 
You may record the match in progress to a file, with the command:
 
You may record the match in progress to a file, with the command:
 
:<tt>record <'''demo name'''></tt>
 
:<tt>record <'''demo name'''></tt>
Line 120: Line 167:
  
 
Automated delayed feeds are supported in ettv beta 8. See the above.
 
Automated delayed feeds are supported in ettv beta 8. See the above.
 +
 +
===Playing back previously recorded ETTV demos===
 +
This is essentially the same as delayed feeds above.
 +
* Configure your server as described in [[ETTV:Server_Configuration#Installation_-_.22Slave.22_.28broadcast_server.29|Slave Installation]] section
 +
* Obtain the <tt>.tv_*</tt> demos and place them in <tt>''fs_game''/demos</tt>
 +
* Start the ettv executable with pb disabled
 +
* <tt>set ettv_autoplay 1</tt> if you want to play back sequential demos named <tt>demo0000.tv_84</tt>, <tt>demo0001.tv_84</tt> etc.
 +
* <tt>demo '''<yourdemoname>'''</tt>
 +
* connect a client and enjoy the show!
  
 
==Troubleshooting==
 
==Troubleshooting==
Line 129: Line 185:
 
* It is a good idea to keep the master server pk3 list to a minimum.
 
* It is a good idea to keep the master server pk3 list to a minimum.
 
* If you are using the autorecord/autoplay feature for running delayed feeds, you will usually want to make sure you clear out the demos directory first. Autorecording will start numbering demofiles using the first available number, so if you have old recorded demos then ettv might start recording from eg demo0017. If your scripts expect to start playback from demo0000 then you might end up broadcasting a game recorded at some earlier date...
 
* If you are using the autorecord/autoplay feature for running delayed feeds, you will usually want to make sure you clear out the demos directory first. Autorecording will start numbering demofiles using the first available number, so if you have old recorded demos then ettv might start recording from eg demo0017. If your scripts expect to start playback from demo0000 then you might end up broadcasting a game recorded at some earlier date...
 +
* If the broadcaster constantly reloads on round starts etc, it's probably a problem with too many pk3s on the ettv servers.
  
 
==Miscellaneous==
 
==Miscellaneous==
 
+
* Slaves may send chats to the master with the command msay
Slaves may send chats to the master with the command msay
+
* Slaves may list players on the master with the command players
Slaves may list players on the master with the command players
+
  
 
==Performance tuning==
 
==Performance tuning==
Line 149: Line 205:
 
:<tt>echo "524288" > /proc/sys/net/core/rmem_max</tt>
 
:<tt>echo "524288" > /proc/sys/net/core/rmem_max</tt>
 
:<tt>echo "524288" > /proc/sys/net/core/wmem_max</tt>
 
:<tt>echo "524288" > /proc/sys/net/core/wmem_max</tt>
 +
 +
These values will be lost on a reboot of your system. A good way to reapply the values upon boot is to include the following in your /etc/sysctl.conf:
 +
 +
:<tt>net.core.rmem_default = 524288</tt>
 +
:<tt>net.core.rmem_max = 524288</tt>
 +
:<tt>net.core.wmem_default = 524288</tt>
 +
:<tt>net.core.wmem_max = 524288</tt>
  
 
FreeBSD can try this:
 
FreeBSD can try this:
Line 160: Line 223:
  
 
'''ettv_netblast''' (default 1)
 
'''ettv_netblast''' (default 1)
:by default, ettv tries to send all fragments at once. some networks may handle this badly. setting this to 0 disables. warning: setting this to 0 without increasing ettv_mtu will result in severely laggy slave playback.
+
:by default, ettv tries to send all fragments at once. some networks may handle this badly. Possible values are:
 +
:* 0 - disable sending all fragments at once. Warning: setting this to 0 without increasing ettv_mtu will result in severely laggy slave playback.
 +
:* 1 - send all fragments at once (default behaviour)
 +
:* 2 - use the old (very broken) beta4 netblast behavior (only applicable for ETTV beta 11 & higher)
  
 
'''ettv_mtu''' (default 1400)
 
'''ettv_mtu''' (default 1400)
 
:; master only
 
:; master only
:if your network / os supports it, you can set this to large values (eg 16000) to prevent message fragmentation and provide smoother ettv links. this cvar has no effect on normal ET clients. note: ettv_mtu must currently be set from commandline, it cannot be changed on the fly (yet).
+
:if your network / os supports it, you can set this to large values (eg 16000) to prevent message fragmentation and provide smoother ettv links. this cvar has no effect on normal ET clients.  
 +
:''note'': ettv_mtu must currently be set from commandline, it cannot be changed on the fly (yet).
  
 
'''ettv_tv_iptos''' (default 0)
 
'''ettv_tv_iptos''' (default 0)
Line 170: Line 237:
  
 
'''ettv_sv_iptos''' (default 0)
 
'''ettv_sv_iptos''' (default 0)
same as ettv_tv_iptos, but affects packets for normal ET clients.
+
:same as ettv_tv_iptos, but affects packets for normal ET clients.
 +
 
 +
===Article Reference===
 +
(x86|exe) refers to the file extension of the ettv file, for linux use .x86 and for windows use .exe
 +
(so|dll) refers to the file extension of the slave file, for linux use .so and for windows use .dll
 +
 
 +
[[Category:ETTV]]

Latest revision as of 08:36, 25 November 2015

This is a graph with borders and nodes that may contain hyperlinks.
About this image

Installation - "Master" (gameserver)

Copy ettv.(x86|exe) to the same directory your etded.(x86|exe) is located. Start your gameserver as normal, but run ettv.(x86|exe) instead of etded.(x86|exe).

Set some cvars

ettv_sv_maxslaves (default 0, no slave connections allowed)

Number of ettv slaves allowed to connect.
Note: Slave slots are always reserved, like private slots.

ettv_password (default "", no slave connections allowed)

Password for ettv slaves.
Must be set, or no slaves will be able to connect!

b_ettv_flags (default 3)

1 = prevent slaves from being callvote kicked
2 = grant shoutcaster status to slaves
values can be added together, eg 3 for both features.

Playing on the master

Players may now connect to your gameserver as normal.

Installation - "Slave" (broadcast server)

Copy ettv.(x86|exe) to the same directory your etded.(x86|exe) is located. Copy tvgame.mp.i386.(so|dll) to the same directory as your etpro/qagame.mp.i386.(so|dll) is located.

Set some cvars

sv_maxclients

Number of spectator clients to allow. This number is only limited by available cpu and memory.
Some guidelines when provisioning a broadcast server:
  • 100 spectator slots will require ~ 128mb ram. Use com_hunkmegs to set this.
  • 100 spectator slots will require ~ 768kbyte/sec (6mbits), roughly 6kbyte/sec per client
  • 100 spectator slots will take ~ 80% cpu of a Xeon 2.2ghz
  • Each slave will require ~20-40kbyte/sec from the master.

ettv_clientname (default "ETTV")

Playername the slave will use when connecting to the master, eg "ETTV-xyz"

ettv_delay (default 0, disabled)

number of seconds for slave to delay the master feed. Recording/playback chains should no longer be needed, and slaves should always maintain an exact delay from the master server.
note: this should ONLY be set on the slave.
note: only supported in ettv beta8 and later

Chat passwords

In order to curb the rampant abuse of ettv chat, admins may set "chat passwords" on their servers. If enabled, viewers are not allowed to chat unless they have a password. You can define as many passwords as you like, so you can easily track down individual abusers and limit the damage of stolen passwords.

ettv_chatpasswords (default 0, disabled)

number of chat passwords defined.

ettv_chatpassword(1-99999)

individual chat passwords.

viewers use /chatpassword <password> to login.

if logged in, ettv chats will be tagged with a number in parentheses identifying which chat password the viewer is using.

Misc commands for viewers

For ettv beta 4 and above
  • spectators may /ignore and /unignore other spectators
  • semiadmin functionality now works for slaves
  • rcon may now mute/unmute specs

Connect the slave

Now start your slave server by running ettv.(x86|exe)

Tell the slave server to connect up to the master server:

tv connect master.ip.address:port ettv_password(required) g_password(if the master has a g_password set)

If you have configured everything correctly, the slave should connect to the master. Clients should now be able to connect to the slave and spectate the game in progress.

This is a "live broadcast" with no delay.

Disconnect the slave

To tell the slave to disconnect from the master:

tv disconnect

Slave Chaining

What is chaining and why chain slaves?

ETTV allows chaining slaves together. This very neat feature lets you connect one ETTV slave to another ETTV slave that is broadcasting a match instead of directly to the game server hosting the match. Apart from the "w00t this is very cool" factor, why is this feature so important for admins? Basicly, working in a chained model allows to broadcast a match to a large number of spectators without too much hastle and too many risks for the master server.

Chaining was developped for a simple reason: if you take into account that one ETTV slave connects at an unlimited rate to a master, you really don't want to connect 15 or 20 slaves to a game server to broadcast let's say the Atlantic Battle. Chances are you will bring down the master game server and thus the game. Chaining slaves firmly alleviates the load on the master game server. 2 or 4 of these Hubs can do the job of delaying the signal and feeding it to other slaves.

Other benefits might include better network performance. If you can choose to either connect your slave to the game server where your connection is not optimal or to a nearby slave who is on a better connection to the master, it might help the quality of your broadcast. Plus you don't need to record anything when connecting to a delayed slave.

Setting up your slave to allow connections from other slaves

This part is actually pretty easy. You need to set up your slave as if you were setting up a master game server. This means that, next to the normal slave parameters, you have to set up the following:

  • b_ettv_flags: set this to at least 1
  • ettv_sv_maxslaves: set this to at least 1. Depending on how powerful your machine is you can set this number higher. Don't set it too high though, a slave still connects at an unlimited rate
  • ettv_password: set to something else than "". This password is needed by the slaves connecting to your slave (together with g_password of course)

Other settings such as ettv_netblast, ettv_tv_iptos etc... can be set as well. You can even use autorecord if you wish, or just add an extra delay.

Pitfalls

There are however a few risks associated with slave chaining. One of the biggest issues is PK3 dependancies.

As you know, a slave needs to have at least all the PK3s of the server it connects too. But a slave can have more PK3s installed, as long as it has the ones on the server's pure list. This is what complicates things! The further you go down the "chain", the more the chances exist of PK3 pollution, which makes the job of the last slave very hard. And the viewers connecting to that slave will not be very happy either.

So for those of you venturing into chaining slaves here's a rule of thumb: always keep the installed PK3s to the bare minimum! Do not add stuff you do not need. If you want to know what PK3s you need and not need, connect your slave to the master or the hub, see which PK3s are on the pure list and rename or remove the PK3s that are not on the list. You will make your viewers and other admins connecting to your slave very happy.

Another risk could be chained lag. If one of the slaves that is higher in the chain starts lagging, the consequenses for the slaves depending on it will be seen immedeatily. So if you offer to be one of the higher slaves in the chain, make sure your machine can handle it! An ETTV chain is much like the food chain ;-).

Admin contributions

If by now you feel like getting your hands dirty in ETTV chaining madness and you want to see some example of hubs, slaves, broadcasters and scripts to get it all working, you can already start by checking out deej's page.

ETTV Demos

ETTV can record and playback ETTV demos. ETTV demos have the extension .tv_84 (or .tv_83 for older versions) and are completely different from the .dm_84 demos recorded on clients.

You may record the match in progress to a file, with the command:

record <demo name>

If you omit the demo name, ettv will start recording using a filename of demoXXXX where XXXX is the first available number starting from 0000.

Demos will stop recording when a map changes.

To stop recording a demo:

stoprecord

To play back a recorded demo:

demo <demo name>

To check the status of an ettv slave:

tv status

This will tell you if the slave is connected to a master, or if it is currently playing back a demo.

To fastforward a demo:

ff <seconds>

Eg ff 30 will fast foward 30 seconds. Fast forward stops at map restarts, so you can safely ff 999 to fast forward through a warmup.

note: requires ettv beta 7 or later


Autorecording demos

ettv is able to automatically continuously record demos, creating new demofiles when the map changes. To do this, set the cvar ettv_autorecord to 1. ettv will start recording demos with the name demoXXXX where XXXX is a number starting at 0000 and incrementing on each map change.

Server admins may find it useful to use the demo recording feature of ettv to record matches, instead of requiring individual players to record them. An ettv demo records everything that happens on a server, including all player viewpoints.

How to create demo based delayed feeds

Autorecording demos is the first half of the process for delayed feeds. Typically you would run a passworded slave server with the smallest number of sv_maxclients possible. No spectators should connect to this server. The only purpose of this slave is to connect up to the master and record the match to sequentially numbered demofiles.

Now start up a slave, and tell it to broadcast the demo being recorded.

demo demo0000

If you wait approx. 60 seconds after the master begins recording before issuing this command to your slaves, then you have a 60 second delay on your broadcast.

If you want slaves to automatically switch to the next demo when the end of the demo is reached (eg on a map change), you would set the cvar ettv_autoplay to 1. If a slave is playing demo0000 and it reaches the end, it will then try to play demo0001 and so on.

Disadvantage of this method: It is not automated in ettv itself, though admins can script it. Advantage of this method: You may run multiple broadcast servers from the same single demo stream.

Automated delayed feeds are supported in ettv beta 8. See the above.

Playing back previously recorded ETTV demos

This is essentially the same as delayed feeds above.

  • Configure your server as described in Slave Installation section
  • Obtain the .tv_* demos and place them in fs_game/demos
  • Start the ettv executable with pb disabled
  • set ettv_autoplay 1 if you want to play back sequential demos named demo0000.tv_84, demo0001.tv_84 etc.
  • demo <yourdemoname>
  • connect a client and enjoy the show!

Troubleshooting

  • The slave server must be running the same fs_game setting as the master server.
  • The slave server must have a copy of every map pk3 the master server is using.
  • If the master server loads a map the slave server doesn't have, the slave will get stuck.
  • Spectators will be forced to download any custom pk3 you are using.
  • It is a good idea to keep the master server pk3 list to a minimum.
  • If you are using the autorecord/autoplay feature for running delayed feeds, you will usually want to make sure you clear out the demos directory first. Autorecording will start numbering demofiles using the first available number, so if you have old recorded demos then ettv might start recording from eg demo0017. If your scripts expect to start playback from demo0000 then you might end up broadcasting a game recorded at some earlier date...
  • If the broadcaster constantly reloads on round starts etc, it's probably a problem with too many pk3s on the ettv servers.

Miscellaneous

  • Slaves may send chats to the master with the command msay
  • Slaves may list players on the master with the command players

Performance tuning

Spectator servers with large numbers of clients (eg >100) will push the OS very hard with huge amounts of UDP traffic. The default OS buffer settings for Linux are inadequate and can hurt performance, causing lag and/or disconnects.

If you get messages like

NET_SendPacket ERROR: No buffer space available to 192.168.120.142:27960

Then you definitely need to increase your kernel network buffers.

Large spectator servers should increase the kernel buffers.

The following commands should be issued as root on linux:

echo "524288" > /proc/sys/net/core/rmem_max
echo "524288" > /proc/sys/net/core/wmem_max

These values will be lost on a reboot of your system. A good way to reapply the values upon boot is to include the following in your /etc/sysctl.conf:

net.core.rmem_default = 524288
net.core.rmem_max = 524288
net.core.wmem_default = 524288
net.core.wmem_max = 524288

FreeBSD can try this:

sysctl -w net.inet.udp.recvspace=65535
sysctl -w net.inet.udp.maxdgram=57344
sysctl -w kern.ipc.maxsockbuf=8388608

Network tuning cvars

The following are only available in ettv beta 7 and later.

By default, et uses small packets of 1400 bytes or less to send to clients. If messages need to be larger, they are 'fragmented' and sent over several server frames. ettv slave messages tend to be large and often fragment. ettv normally tries to send slaves all fragments at once to avoid jittery playback, but some networks / OSes handle this badly. ettv beta 7 introduces new controls so admins can optimize ettv for their individual network.

ettv_netblast (default 1)

by default, ettv tries to send all fragments at once. some networks may handle this badly. Possible values are:
  • 0 - disable sending all fragments at once. Warning: setting this to 0 without increasing ettv_mtu will result in severely laggy slave playback.
  • 1 - send all fragments at once (default behaviour)
  • 2 - use the old (very broken) beta4 netblast behavior (only applicable for ETTV beta 11 & higher)

ettv_mtu (default 1400)

master only
if your network / os supports it, you can set this to large values (eg 16000) to prevent message fragmentation and provide smoother ettv links. this cvar has no effect on normal ET clients.
note: ettv_mtu must currently be set from commandline, it cannot be changed on the fly (yet).

ettv_tv_iptos (default 0)

you can set the ip type of service field of ettv link packets. if your network/ISP supports it, you can use a value like 16 ("minimize delay") to prioritize ettv link packets over others on the network.

ettv_sv_iptos (default 0)

same as ettv_tv_iptos, but affects packets for normal ET clients.

Article Reference

(x86|exe) refers to the file extension of the ettv file, for linux use .x86 and for windows use .exe (so|dll) refers to the file extension of the slave file, for linux use .so and for windows use .dll