Feature #5520

Implement Addon Syncronization

Added by oktane almost 6 years ago. Updated over 2 years ago.

Status:Assigned Start date:10/28/2009
Priority:Normal Due date:
Assignee:Dwarden % Done:

0%

Category:Multiplayer
Target version:NextGen
Affected ArmA II version: First affected ArmA II version:
Reproduced by another DH user:No Single / Multi Player?:
I am using some Mods:No BIForumURL:
I am using: NGUrl:
Reproducible for you:No WIKIurl:
Related to content of DLC:

Description

The game could synchronize addons (caching them in A2 temp folder?) when it does its mission download, as other modern games do. This would allow the server admin to choose quality community modifications and missions that require them, and not have to suffer the loss of players on a public server. As it is now, when a player joins without the addons, he is kicked off with a pretty non-descriptive message. We have no way to communicate to the kicked player where to download the required modifications, so he goes somewhere else or gives up. The only avenue of communicating information to a player that cannot join is the server name itself, which then becomes cluttered.. this is not ideal.

The game could use calculated CRC's to determine if it needs to download them, or a number of other methods. (add a field in the config.cpp that is a version number?)

The game need not serve the files directly, instead a hosting url should be specified in the server.cfg to point to where the client should download the files from, preventing lag for the people already on the server.


Related issues

related to ARMA2 Community Issue Tracker - Feature #5523: Multiplayer Kick Reasons - Need customizablity Assigned 10/28/2009
related to ARMA2 Community Issue Tracker - Feature #21591: Per mission config space modification. Assigned 06/30/2011
related to ARMA2 Community Issue Tracker - Feature #17496: Provide a simple interface to support community addon dow... Assigned 02/09/2011
duplicated by ARMA2 Community Issue Tracker - Feature #18523: Download System for DLC Duplicate 03/27/2011 04/08/2011
duplicated by ARMA2 Community Issue Tracker - Feature #6573: download addons Duplicate 12/06/2009

History

Updated by kju almost 6 years ago

  • Status changed from New to Assigned

Updated by ModeZt almost 6 years ago

The idea is great.
But currently the game requires a restart to load a new update. Onthefly addon loading has to be implemented to make it work.

Updated by Whisper almost 6 years ago

Having the game downloading addons then restart with correct addons loaded is OK, if you ask me. Plenty of games do like that already.

This feature would be a MAJOR plus

Updated by barmyarmy almost 6 years ago

This would be a huge plus and something of a necessity if Arma is ever to become more mainstream; at the moment it requires a dedicated enthusiast to get to grips with downloading and installing mods.

Its staggering this has never been included before now.

Updated by ModeZt almost 6 years ago

but the addons can be unique on different servers.
1. you try to join a server that requires addon(s)
2. addon(s) is(are) automatically downloaded
3. game restarts with this addon(s)
4. you don't remember the server you were trying to join.. you select a new server and goto step 1

I agree that all the addon installation and launching is a bit too difficult. But this fully-automated thing won't work without the OnTheFly addon loading.

On the other hand remember ACE1 for Arma1. The installation, launching and updating was really user-friendly..

Updated by Whisper almost 6 years ago

the game restart and connect to the same server by itself.

(See Ennemy Territory, for example)

the option to connect to server X on ArmA2 startup already exists, if I remember well

Updated by Yoma almost 6 years ago

Obviously a great idea.
However there are many caveats in it's implementation.
-It should have 0 effect on server/client bandwidth (people complain of lag already enough as is)
-I personally think it's a better idea to use something outside the game itself (external tool), make it a launcher as well, something along the lines of my tool, but shipped with the game. Give it support for Http/FTP/P2P/something rsync alike for downloads and allow server admins to take it in whichever direction they want to take it.
-My tool can be both a reference to how to do and not to do stuff
-Another even larger idea could be to set up a P2P mesh where addon makers can submit their works to, set up some kind of addon validation system to allow uploads to the mesh, both freedom and quality are key.
-Implement internal dependency checking for addons (integrated hashes inside addons?) to resolve dependencies.
-The system could also be used to verify/fix user installations (great for people who tend to screw up patches etc) it could even replace the entire patching procedure by something differential file based.
-Remember that some people do not like P2P.
It's all possible but would require deep dedication and quite some funds to implement well I think.

Updated by oktane almost 6 years ago

I don't mind if the game has to restart, but good point.. I had forgotten about that. The game needs to restart because it parses config.bin/cpp of loaded addons on start.

Yoma, concerning bandwidth, that is why I specifically mentioned that there has to be a way to configure WHERE the addons are in the server cfg. (an external web host is where I would put them, well away from the game hosting box)
P2P mesh would be cool although logically it would only work want to connect with the other people on the server so their bandwidth would be affected, which is bad.
Verification is good, based on CRC checksums/signatures. And yes, people would end up blocking P2P.. plus without an 'standalone' client/server (like uTorrent) running, only people running the game could contribute (seed), and because of the fact they are running the game and online, must mean their bandwidth would be affected. I don't know many people that leave the game running at the menu for giggles, so the only seeders would be the people playing.

We need ANYTHING really, even if it's later a 3rd party application hook, api call, customized message box, we just need a way to address this simple issue: New players to the game cannot join some servers with addons and they have no idea what to do. Servers that are receptive to new people (like ours) want to accommodate them and teach them the game but from the start they need the missing information to even get in the server to talk to us. Therefore we are stuck playing vanilla to accommodate them, which is a major bore. A secondary problem that exists even with experienced players is addon versioning, they don't have the right version etc.

If we just had some kind of system where the addons could be downloaded by the game, think of all the problems it could solve like crashing caused by bad addons, low player counts because of frustrated new people, more variaty and faster introductions of 'quality' mods throughout the community! If xyz server hosts JoeBlow's explosion effects and you see how awesome they are, well wow, now you have a new mod you want to use when you play anywhere! (which you will configure yourself)

Arma is like a teenage kid who doesn't want to talk to is parents and tell them whats wrong. :D It isn't giving players enough information, usability wise.

Whatever the solution is, a basic version of it must come with the SHIPPED game.

Updated by zGuba almost 6 years ago

  • Affected ArmA II version deleted (Please select...)

This would need game beign able to change config on the fly.

Updated by zGuba almost 6 years ago

  • Operating system deleted (Win7 64 bit)
  • Graphics card deleted (n/a)
  • Graphics card driver version deleted (0)
  • Maxmem parameter deleted (Not set)
  • Difficulty deleted (Not set)

Updated by oktane almost 6 years ago

As a note, even if this can't be or isn't implemented, more descriptive messages would help. That item is here http://dev-heaven.net/issues/show/5523 So please if you vote for this one, vote on the other. Thanks

Updated by kju almost 6 years ago

  • Target version deleted (331)

Updated by oktane almost 6 years ago

zGuba wrote:

This would need game beign able to change config on the fly.

I think most are fine with it restarting after downloading. Some other games do that. It could just reconnect with the command line parameters -connect etc. I fully respect the effort required that even the simplest of implementations of this would take, hence the suggestion that makes it easier on BIS.

Updated by zGuba almost 6 years ago

oktane wrote:

zGuba wrote:

This would need game beign able to change config on the fly.

I think most are fine with it restarting after downloading. Some other games do that. It could just reconnect with the command line parameters -connect etc. I fully respect the effort required that even the simplest of implementations of this would take, hence the suggestion that makes it easier on BIS.

Hmm. I've noticed that binarized models contain config that can be overriden by screwed models depending on loading order, so it's not that hard after all.

Updated by CarlGustaffa almost 6 years ago

I'd rather have this outside Arma, in a separate launcher. If Arma forced me to wait until CAA addon was downloaded, well, you get the picture :) Many addons these days are huge. If inside Arma, with automatic restart, I would want to:
  • Be able to choose to continue downloading or go back to server list.
  • Be able to choose i.e. between allowed addons I already have, or those suggested by the server.
  • Cross reference any conflicting addons. If I have addon A, I should be warned that it conflicts with addon B located on the server.
  • Control the priority of addon loading (-mod=@A;@B etc).

In general, too much hazzle to deal with internally than an external app developed by the community who better knows what is needed once we identify the problems.

Updated by Anunnaki over 5 years ago

Steps:

1) Someone is connecting to the server ... server returns the XML file with list of every addon file, which is necessary to connect to the server. The files would be divided to the groups, where every group have minimum one or more download sources. Every file must have: unique ID (MD5/CRC32 hash), size, name, and of course "destination mod folder".
2) Than clint application must have one folder dedicated as "temp-repository" and there XML file with the same type of informations, like a normal server have. But in this file are stored all informations about addons=files in this "temp-repository". Client than compare, which file he already have in theyr temp-repository, and those just copy to the "folder structure", which the server needs. And all files missing on the client, he will automatically download to the "temp-repository" and to the "mod folders" too, when needed.

That it is. Players can play on many servers with some different and some the same mods. The "client" will prepare "cooks" all necessary mod folders and copy the needed files there from local repository. And when something would be needed more, it can be downloaded from URLs received from gameserver, which need those files.

And perhaps to the future, there can be alternative dedicated BIS "database" server, which can contain this XML database, about many version of files and URL addresses, where client's applications can connect to search alternative URL address, to download necessay files.

Updated by oktane over 5 years ago

In general, too much hazzle to deal with internally than an external app developed by the community who better knows what is needed once we identify the problems.

Sigh.
The simplest implementation is required in the core game itself though. Try being a server admin for a few years.. your regulars want new addons to freshen the gameplay, but new users cannot join your server. This is the core issue. External apps will not work because new users A) have no idea about them B) we cannot even tell them about an external app via the game. (ie, popup message when unable to join the server saying 'go to this website and get this app') Besides that, it's jumping thru more and more hoops!

Yoma's already does what you ask Carl, aside from crossref which would require a compatibility db. But not a single external app helps people new to the game who wish to join a server. The game will always be un-popular when it's so hard to use for newbies, hence less people on multiplayer.

Updated by Meek about 5 years ago

Garry's Mod is definitely something to look at, it handles mod syncing really well.

I dislike the idea of elaborate systems involving P2P and external applications, files should be downloaded from the server being connected to, this is the way things have been done since nearly the start of multiplayer FPSes (I remember Quake 1 doing this). It is a time-proven method of downloading modded files, and is still used today.

When a user connects, it grabs a list of required files and their hashes from the server, it scans over every file in the list, looking to see if there is a file with the same hash (files will be stored locally using the hash as the filename), if the file does not exist locally, download it.

This is repeated in every file in the server-provided list, eventually all the required files will be on the client's machine.

The benefit to this is that playing on other servers that use the same files won't cause you to download every file again.

Updated by kju about 5 years ago

To allow download from a different server is quite useful though.

Updated by INNOCENTandCLUELESS over 4 years ago

Let's summarize:

- the users client should download addons which are defined within the mission at the server
- the users client should be configurable so that immediatede use of addons or DL after confirmation is possible
- the mission at server should contain data where to ask/where to DL addons

conclusions IMHO:

- to make this work BIS must define a much strikter framework how a mission must look like, containing definitions
- to avoid fraudulent usage a trust center is required to avoid that this infrastructure will be highjacked to inject malware or to steal user data
- BIS must take over a certain portion of responsability

Hence: it will never happen that BIS will do this, they will rather code necessary functionality.

Most concerning issue is that tools like SixUpdater open your PC and bypassing the FW.
Even if the tool itself is trusted, it is still possible that malware appears at the listed servers.

Updated by Sickboy over 4 years ago

INNOCENTandCLUELESS wrote:

Most concerning issue is that tools like SixUpdater open your PC and bypassing the FW.
Even if the tool itself is trusted, it is still possible that malware appears at the listed servers.

Generally you're synchronizing addon pbo files, readme's, etc, with only the exceptions jayarmalib and acre where dll's are involved, or beta's where exe's are involved - while these can be ignored.
This is just downloading, like your web-browser does, etc, so im personally not seeing the argument of 'bypassing the firewall and opening your pc up'.
Being allowed to download through the firewall also doesn't excempt from antivirus/malware protections etc.

But, regardless, we could try to put effort into running without admin rights - which should be easy for the most part (give arma2 folder proper permissions or use a mods folder in user-space), but things like ACRE require running as admin due to their pipe ops...
#18122

In any case, if a full bis solution is too much work/time involved, I would vouch for API in the game that we can use to do updating tasks etc, this way BIS can keep a handle on it, and they don't have to spend a lot of work / time.

Updated by Dwarden over 3 years ago

  • Assignee set to Dwarden
  • Target version set to NextGen

for the moment this is more nextGen than thisGen

Updated by luckzeh over 2 years ago

With Steamworks this could be more of a possibility?
(Though I have to say the multitudes of third party tools that sprung up for Arma 2 work well enough)

Also available in: Atom PDF