Bug #26323

Issues with v2 signatures - causing random pbo's / files in pbo's to fail

Added by admin over 3 years ago. Updated over 3 years ago.

Status:Closed Start date:11/13/2011
Priority:Normal Due date:
Assignee:ViperMaul % Done:

0%

Category:Multiplayer
Target version:1.60.87580
Affected ArmA II version:1.60 BETA First affected build:Please Specify...
Reproduced by another DH user:No First affected ArmA II version:1.60 BETA
I am using some Mods:No Single / Multi Player?:
I am using: BIForumURL:
Reproducible for you:No NGUrl:
Related to content of DLC: WIKIurl:

Description

  • Wrong signature for file ca\rocks2\r2_boulder1.p3d
  • Wrong signature for file C:\Program files\Bohemia Interactive\ArmA 2\Expansion\beta\addons\ui.pbo

Related BIF post; http://forums.bistudio.com/showthread.php?p=2055020#post2055020

logs.zip (385.1 kB) ArMaTeC, 11/15/2011 17:21

ATOW_MEPS_Server_Logs.rar (231.8 kB) ViperMaul, 11/15/2011 22:43

Client_ViperMaul_Arma2_checksums_2.md5 (101.8 kB) kju, 11/16/2011 05:12

ATOW_SERVER_ArmA_2_MEPS_2.md5 (137.9 kB) kju, 11/16/2011 05:12

logs.zip (256.6 kB) ArMaTeC, 11/16/2011 14:51


Related issues

related to ARMA2 Community Issue Tracker - Bug #24568: Certain random PBO's fail signature checking with latest ... Closed 09/16/2011

History

Updated by Sickboy over 3 years ago

  • Description updated (diff)

Seems random pbo / files (inside pbos) failing, when certain addons / bisigns are introduced in the mix.

I have seen this behavior before, when joining with 1.59 or old 1.60 betas to a newer 1.60 beta server, even with v1.
Best test with the latest available Beta because it is bumped to v1.60 and thus older betas cannot join

Seems related to earlier issues like #26323 - perhaps not all conditions are solved.

Probably caused by specific pbos - Best to enable modfolders one by one and pinpoint the pbo(s).

Updated by Sickboy over 3 years ago

  • Description updated (diff)

Updated by kju over 3 years ago

Requires hashes of both client and server pbos, and server.cfg!

http://code.kliu.org/hashcheck/

Updated by ViperMaul over 3 years ago

I can confirm this using mod files from the AToW tournament
Also ArMaTeC can confirm using his City Life mod.

My experience is the following...

== "C:\Program Files\Bohemia Interactive\ArmA 2\Expansion\beta\arma2oa.exe" "-mod=Expansion\beta;Expansion\beta\Expansion;@atow-cba;@AToW-C7;@ATOW-Islands;@ATOW-JSRS;@ATOW-Debug" -name={UN}ViperMaul -noFilePatching -showScriptErrors -nosplash -world=empty
..
Version 1.60.86277

which signs their files with v2 sig tools using http://dev-heaven.net/issues/24568#note-26

I had to use the server net.log to get the following information which was difficult.
My first attempt - Player {UN}ViperMaul: Wrong signature for file ca\characters_e\un\un_cdf_officer.p3d
My second attempt - Player {UN}ViperMaul: Wrong signature for file cnr\cnr_krizov\cnr_krizov.wrp
My third attempt - Player {UN}ViperMaul: Wrong signature for file ca\rocks2\r2_boulder1.p3d

Other players got different files as well
Player [DI]I|SSgt.Sprite: Wrong signature for file ca\data\penetration\wood.rvmat
Player [IRT]PVT.Woodstock: Wrong signature for file baf\addons\data_baf.pbo
Player [IRT]PVT.Woodstock: Wrong signature for file C:\Program Files (x86)\Steam\SteamApps\Common\ArmA 2\addons\plants2_tree.pbo

ArMaTeC's uses the following...

== "D:\Game Server\Bohemia Interactive\ArmA 2\Expansion\beta\arma2oaserver.exe"  "-profiles=default" "-exThreads=7" "-CpuCount=4" -mpStatistics "-nosplash" "-skipIntro" "-world=none" "-mod=Expansion\beta;Expansion\beta\Expansion;@CBA;@CityLife2Core;@CityLife2;@CityLife2Com" -noland -netlog "-BEpath=default\BattlEye" "-config=default\server516516516516.cfg" "-cfg=default\Arma2.cfg" "-name=server516516516516" "-port=2366" 
..
Version 1.60.86277

He gets the same issues.

Our server.cfg is using

verifySignatures=2;
BattlEye = 0;
//Cheat detection
checkfile=0; //1=slow 0=defult
onHackedData = "kick (_this select 0)"; 
onUnsignedData = "kick (_this select 0)";     // unsigned data detected
kickDuplicate=1;        // do not allow duplicate id

Updated by ViperMaul over 3 years ago

My Steps To Repo

1. Use verifySignatures = 2; and use the -netlog (so you can get the specific file)
2. Log in to slot selection screen.
3. Start mission
4. Attempt to get to Map Briefing Screen. (usually you will notice it here)
5. Get boots on ground and wait 30-60 seconds.

Result "Session Lost" dialog
Then look at the server side net.log as this is the only place that shows which files you got booted from.

Updated by kju over 3 years ago

Again

Requires hashes of both client and server pbos, and server.cfg!

http://code.kliu.org/hashcheck/

Updated by Dwarden over 3 years ago

ideal case, upload the offending files for both server and client (hopefully the file is small)

Updated by Sickboy over 3 years ago

Yep, md5 sums of the whole game has little use I think - it works fine with sig check v1, and the random pbo's failing should not be the actual signature problems - some 3rd party addon(s) should cause it instead or in my case in the past it was issues with older beta files; http://dev-heaven.net/issues/24568#note-10

Also im getting some signals that md5sums of several files are different per distribution :O

ViperMaul - did you verify on the clients experiencing the issues that the beta was cleanly installed? So deleted the old beta files before updating to the latest?

What about the PMC/BAF expansions - I think you mentioned those before - do they actually have the correct v2 bisigns?

Updated by ViperMaul over 3 years ago

Sickboy wrote:

ViperMaul - did you verify on the clients experiencing the issues that the beta was cleanly installed? So deleted the old beta files before updating to the latest?

Yes. In three ways:
  1. Using the new #userlist which reports which build they used
  2. Reviewing their RPT and mine.
  3. And in latest tests using 86277 which earlier build would not be able to connect due to the 1.60 requirement.

What about the PMC/BAF expansions - I think you mentioned those before - do they actually have the correct v2 bisigns?

I did that my personal client. The only way I know how is the following:
  1. I looked at the size of the bi2 sigs for the Arma2 Addons, common and DTA folders including in the expansion beta folders and they were ~560 bytes. My understanding is v1 sigs are ~160 bytes in size. Using this size method they all checked out.
  1. I also used dschecksignatures.exe with the -deep param
    dschecksignatures.exe -deep "D:\Arma2_ArrowHead_Server\ArmA 2\addons" "D:\Arma2_ArrowHead_Server\ArmA 2\keys" 
    

    I got no errors in the results. I didn't get the chance to run that command on other areas like Common, DTA or Expansion yet.
  1. ArMaTeC and I used the Six Updater's Game Manger and I got some of the following results. However I don't know how the Game Manger internals works yet so you might be able to give some commentary the validity of the results.
The Game Manager shows many Bi signs failing and many Bi2 passing (for ArMaTeC)
For (ViperMaul) The Game Manager shows even Bi2's failing in some areas like DTA
For (ViperMaul) The Game Manager shows all passing for PMC files, Common files, and Expansions files for me (both bi and bi2 sigs).

Updated by Sickboy over 3 years ago

ViperMaul wrote:

Yes. In three ways:
  1. Using the new #userlist which reports which build they used
  2. Reviewing their RPT and mine.
  3. And in latest tests using 86277 which earlier build would not be able to connect due to the 1.60 requirement.

That are 3 ways that have nothing to do with what I wrote.
A clean install means deleting the beta folder before installing the new beta.
A dirty install is when you leave the existing files, and then install the latest beta. Aka, you can end up with files that were included in that older beta, but not in the newer. - Very real possibility especially if the user had an old beta installed from before v1.60 betas etc.

I looked at the size of the bi2 sigs for the Arma2 Addons, common and DTA folders including in the expansion beta folders and they were ~560 bytes. My understanding is v1 sigs are ~160 bytes in size. Using this size method they all checked out.

You did the same for the PMC/BAF folders? That was the queastion :P
Did you establish yet if the people experiencing the issues, all have BAF/PMC or all dont have?

Also the -deep check afaik does not verify if a bisign is v1 or v2 I think, just if the sig (v1/v2) is valid for the file.

As to how Six Updater Game manager CRC check works:
  • The files are checked versus md5sums from my installation
  • Perhaps the differences are in the distribution of the game, mine is dvd disk version of both a2 and oa - I made the md5sums on my 1.59 installation
  • PMC and BAF are not checked (you can review the file list and find that there are no PMC/BAF listed) - hard to check as they are encrypted and different for people afaik. :)

Updated by Sickboy over 3 years ago

BTW am I the only one who wonders why random pbo's fail signature checks, while these have valid signatures and work fine when not mixed with certain other addons ?

AKA, addonY is fine, you introduce addonX to the mix, and suddenly addonY is listed as wrong signature. AFAIK that is in essence what is happening here.

First thing is to identify these addons specifically, however.....

How can certain addons interfeir with the results of other (valid) addons?

Updated by Suma over 3 years ago

addonY is fine, you introduce addonX to the mix, and suddenly addonY is listed as wrong signature

I understand this is confusing and it makes creating a minimal repro a bit more difficult, but it still should be possible. Once you have a repro which works and you know it will no longer work if you remove a single addon, it is a minimal repro, even if the error message is about a different addon.

How can certain addons interfeir with the results of other (valid) addons?

In my experience it is most often not worth speculating about bug causes. With a solid repro you can usually see the error cause in the debugger very quickly. Once I know the cause, I will report it back to you.

Updated by Sickboy over 3 years ago

Agreed. On the other hand, in my experience, if you don't have a repro but do know behavior details, then accessing memory or reviewing the code could result in sufficient insight to locate the culprit and produce a fix.
That probably works better on smaller projects though :)

Hopefully ViperMaul will be able to narrow it down to a specific mod folder or pbo (set).

Updated by ViperMaul over 3 years ago

A clean install means deleting the beta folder before installing the new beta.

I do delete the beta folder each time I create our custom Six Updater repository.
But it is true we update the clients using "Install and Update" which my understanding does not remove any old files that should not be there. My understanding is only "Verify and Repair" does this. For this case however, I will make sure the clients manually delete the beta folder and manually install the beta and retest.

First thing is to identify these addons specifically

In between work here, I will work to narrow it down to a single mod folder. I only tried vanilla once due to limited time and it was not reproducible but I didn't have enough players to test with me on that day. I will report back with more.

ideal case, upload the offending files for both server and client (hopefully the file is small)

Dwarden's request is hard because it is never the same file that gets rejected. And most file types are individual files that are packed inside of PBOs like *.P3D files and *.WRT files. I am pretty sure you don't want me to attach these files.

Requires hashes of both client and server pbos, and server.cfg!

Also I am almost finished creating kju's hash request.

Updated by Sickboy over 3 years ago

Dwarden's request is hard because it is never the same file that gets rejected. And most file types are individual files that are packed inside of PBOs like *.P3D files and *.WRT files. I am pretty sure you don't want me to attach these files.

Since the reported signature problems seem to be random / other files than the actual offending files, I think what is meant or should be meant is that once figuring out which modfolder / addon is causing the culprit - to upload those files.

SU stuff:
Wasn't sure you were using SU for the testing of expansion/beta.

I do delete the beta folder each time I create our custom Six Updater repository.

Good - unless you also delete the .rsync subfolder from the custom repo preparation folder - this folder should remain in tact (as it describes repository version, guid etc).

But it is true we update the clients using "Install and Update" which my understanding does not remove any old files that should not be there. My understanding is only "Verify and Repair" does this.

Almost correct. Check out the new FAQ item: http://six.dev-heaven.net/wagn/Six_Updater+FAQ#What's%20the%20difference%20between%20%22Verify%20and%20Repair%22%20and%20%22Install%20or%20Update%22

Updated by Sickboy over 3 years ago

I've been unable to reproduce it locally with the pbos for CityLife's server (ArmaTec).
But this was with deddy running from the same machine+installation as the client, now trying it from other server in LAN.
http://www.pastie.org/2863089 and verifySignatures = 2;

Updated by bebul over 3 years ago

We have prepared Beta Patch rev.86364 containing specific Signature Checking Diagnostics.
Can you run this new server with netlog and give us the feedback?
(The wrong signature lines should look the same as before but there should be some other info in the (...) at the end.)

Updated by ArMaTeC over 3 years ago

here ya go left server running for a while with sign ver 2

Updated by ViperMaul over 3 years ago

Bebul, here is my logs using the latest beta as requested.
Kju, I have hash files as you requested for both client and server. NOTE: They represent my Sprocket A2 and OA versions.

Bebul, something interesting to note.

When I use the AToW mission, one with mods listed in the mission.sqm for example, it produces the problem. http://pastebin.com/EmnvH0Ht
When I use some bare bones mission that has next to nothing for addOnsAuto[] it works successfully no problem. And following that I can use #missions and change back to the AToW mission and this time get in without being kicked. You can follow some of this progression in the net.log and server side RPT file attached.

After I attached the logs I decided to try that again, 1st starting the bare bones mission, no kick, then going into the AToW mission, no kick. But if I disconnect from the server and go back in directly to the AToW mission, I get kicked by this issue. Sometimes, I actually have to restart my client from cold and then go in directly to the AToW mission and only then I can reproduce the kick.

Updated by kju over 3 years ago

The hashes look OK.

To remove the rar and tmp files might be useful though.

Updated by bebul over 3 years ago

The new uploaded netlogs has told us more, but we need yet another, more detailed, feed back.
The new Beta Patch (rev.86411) contains more netlog details in its server implementation.
Can you provide it for us, please?

Updated by bebul over 3 years ago

ViperMaul wrote:

Bebul, something interesting to note.

When I use the AToW mission, one with mods listed in the mission.sqm for example, it produces the problem. http://pastebin.com/EmnvH0Ht
When I use some bare bones mission that has next to nothing for addOnsAuto[] it works successfully no problem. And following that I can use #missions and change back to the AToW mission and this time get in without being kicked. You can follow some of this progression in the net.log and server side RPT file attached.

Looks really interesting. I will try to follow this hint! Thnx!

Updated by bebul over 3 years ago

The issue was understood while tracking the ViperMaul hints, thank you!
Hopefully fixed in rev.86431 [in order to test it properly, the additional netlog debug lines are still active].
Beta patch would come soon ;-)

Updated by Sickboy over 3 years ago

You guys are the best :)

Updated by ArMaTeC over 3 years ago

i know its kinda invalid but this is the logs from the current beta

Edit:

New beta seams to have fixed the issue with sign V2 8 players online and no kicks if you still require logs let me know

Updated by Dwarden over 3 years ago

  • Status changed from Feedback to Resolved

preliminary marking as resolved , if it's wrong raise Your voice ;)

Updated by ViperMaul over 3 years ago

Confirmed fixed on my end.
I can no longer reproduce this issue.
Great work everyone and thank bebul!!

Updated by Dwarden over 3 years ago

  • Status changed from Resolved to Closed

with crossed fingers (and buzz ovations to Bebul in background) closing as fixed ...

Updated by Dwarden over 3 years ago

  • Target version set to 1.60 BETA

Updated by Dwarden over 3 years ago

  • Category set to Multiplayer

Updated by kju over 3 years ago

  • Target version changed from 1.60 BETA to 1.60.87580

Also available in: Atom PDF