Issue Details (XML | Word | Printable)

Key: LIBOMV-420
Type: Bug Bug
Status: Resolved Resolved
Resolution: Won't Fix
Priority: Major Major
Assignee: John Hurliman
Reporter: Anton Lauridsen
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
libopenmetaverse

[patch] the asset download handler assumes that all but the last packets will always be 1,000 bytes long

Created: 09/Nov/08 03:19 PM   Updated: 10/Nov/08 03:51 PM
Component/s: Assets
Affects Version/s: None
Fix Version/s: 0.6.0

File Attachments: 1. File Dump1 (21 kB)
2. File patch2 (3 kB)

Issue Links:
Related
 

Severity: High
Environment: All
Steps to Reproduce: attempt to download any texture asset from an OpenSimulator region.


 Description  « Hide
In AssetMananger in TransferPacketHandler it is assumed that the transferred data will always be 1,000 bytes long, except for the last packet. In OpenSimulator a packet is 1,100 bytes long. This causes texture assets to be corrupted in the byte array buffer, which eventually leads to a seg fault when trying to decode the jpeg 2000 format to the internal format.

The attached patch replaces the current implementation with an improved, but more memory intensive, which will handle wildly out of sequence packets and packets of varying size



 All   Comments   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Jim Radford added a comment - 09/Nov/08 04:28 PM
Hi Anton, Thanks for the information - I'm looking into this with the opensim folks to see why they are deviating from the 'standard' for any particular reason.

Unless there is a compelling reason to do so, we try and stick to the wire standard (ie: we implement the protocol as per the working Linden Grid), deviating from that to support opensim or other implementations is unlikely as it would quickly become a nightmare to manage and support all the different clone grids people are starting to come out with.

Jim


Anton Lauridsen added a comment - 10/Nov/08 02:55 AM
Hi Jim

Just as with 419, I did spend a little time contemplating if I should report this to the OpenSim guys or to you. I decided on the latter due to the following reasons:

1) Different simulators might possibly have different package sizes due to optimizations, especially in very customized grids, handling scenarioes where the first package received is actually the very last in the stream, i.e. it does not have a full buffer of std. size can be trouble some, the likelihood of this happing grows inversely proportional to the size of the asset.
2) The issue caused a segementation fault in the OpenJPEG library, opening an external code execution vulnerability.
3) The comment above mentioned that "for now it appears to be safe to assume 1,000 byte sized packages.

Of course it is all up to you to decide if you accept the patch, you have done a great job so far.


Teravus added a comment - 10/Nov/08 10:20 AM
Hey there:

Just to let you know.. I seem to be able to download texture assets from OpenSimulator regions using LibOMV.

Here's an example: http://teravus.wmcv.com/googletester/idealistviewer20.jpg


John Hurliman added a comment - 10/Nov/08 10:21 AM
Applied in r2343, thanks! This is an unfortunate patch due to the additional and memory copies and fragmentation that will happen, but really just one more reason we need to move away from udp texture transfers entirely.

John Hurliman added a comment - 10/Nov/08 10:40 AM
Just talked with OpenSim, and they correctly calculate transfer packets at 1000 bytes (and the first packet at 600 bytes):

size = m_asset.Data.Length - 600 - (1000 * (PacketCounter - 1));
...
if (size > 1000) size = 1000;

Not sure where you are seeing the 1100 byte number, but it may be from an old revision of OpenSim or a protocol-breaking fork. Rolling this patch back to the old behavior.


Anton Lauridsen added a comment - 10/Nov/08 11:10 AM - edited
I just ran a few tests, I have a standalone open sim region at revision 7227
and it gave me the attached SIGSEV dump (dump 1), regardless of how OpenSim is implemented, I don't think that a SIGSEV is a safe implementation as it leads to some nasty eploits.

I then tried the same codebase on my region running as part of OSGrid, running on r 7217 and got pretty much the same result:

public void ConvertToStream()
{
if(_asset.GetType() == typeof(OpenMetaverse.AssetTexture))

{ AssetTexture texture = (AssetTexture)_asset; _rawData = _asset.AssetData; texture.Decode(); Console.WriteLine("Width="+texture.Image.Width.ToString()+" Height="+texture.Image.Height); _rawData = texture.Image.ExportTGA(); }

}

the segmentation fault occurs when I invoke texture.Decode()


Anton Lauridsen added a comment - 10/Nov/08 11:11 AM
Dump from crash

Teravus added a comment - 10/Nov/08 01:02 PM
Can you give any more details about the environment you're running?

Anton Lauridsen added a comment - 10/Nov/08 01:22 PM
Anything you want

Ubuntu 8-10 64 bit
Mono JIT compiler version 1.9.1 (tarball)
64 bit version of OpenJpeg and with my supplied patch the segmentation fault dissapears, and I can convert the j2p fine.

Without it, the transferred texture contains zero'es at the end of the file, matching the effect of transferring an 1,100 block size would have.


John Hurliman added a comment - 10/Nov/08 03:19 PM
I'm still wondering where the 1100 number came from. Other libomv-based applications connecting to OpenSim are working, and an OpenSim developer confirmed that the 1100 number doesn't show up in the OpenSim source code. I agree that your diagnosis of the problem seems accurate, but what led you to that specific number?

Anton Lauridsen added a comment - 10/Nov/08 03:51 PM
I'm using MonoDevelop as my platform, and there is no debugger !!! ... so I eventually made a debug build of libOMV, using the line numbers in the exception debug log, I finally went in and reactivated the debug logging code which is out commented in that section of the code. It is a bit outdated, but a little tweaking made it work alright!!!

original code -->
//Client.DebugLog(String.Format("Transfer packet {0}, received {1}/{2}/{3} bytes for asset {4}",

// asset.TransferData.Packet, asset.TransferData.Data.Length, transfer.Transferred, transfer.Size,

// transfer.AssetID.ToString()));

<-- original code
new code -->
Logger.DebugLog(String.Format("Transfer packet {0}, received {1}/{2}/{3} bytes for asset {4}",

asset.TransferData.Packet, asset.TransferData.Data.Length, transfer.Transferred, transfer.Size,

transfer.ID.ToString()));

<--new code

gave this result
65227 [7] DEBUG - Transfer packet 2, received 1100/1100/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65227 [8] DEBUG - Transfer packet 1, received 1100/2200/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65228 [3] DEBUG - Transfer packet 0, received 1100/3300/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65228 [9] DEBUG - Transfer packet 4, received 1100/4400/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65228 [8] DEBUG - Transfer packet 5, received 1100/5500/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65228 [7] DEBUG - Transfer packet 6, received 1100/6600/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65228 [9] DEBUG - Transfer packet 7, received 1100/7700/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65228 [9] DEBUG - Transfer packet 3, received 1100/8800/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65364 [3] DEBUG - Transfer packet 8, received 1100/9900/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65365 [8] DEBUG - Transfer packet 9, received 1100/11000/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65365 [8] DEBUG - Transfer packet 10, received 1100/12100/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65365 [8] DEBUG - Transfer packet 11, received 1100/13200/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65365 [8] DEBUG - Transfer packet 12, received 1100/14300/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65365 [8] DEBUG - Transfer packet 13, received 1100/15400/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65365 [8] DEBUG - Transfer packet 14, received 1100/16500/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65507 [3] DEBUG - Transfer packet 15, received 1100/17600/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65508 [8] DEBUG - Transfer packet 16, received 1100/18700/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65508 [9] DEBUG - Transfer packet 17, received 1100/19800/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65508 [3] DEBUG - Transfer packet 18, received 1100/20900/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65508 [8] DEBUG - Transfer packet 19, received 1100/22000/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65508 [9] DEBUG - Transfer packet 20, received 1100/23100/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65509 [3] DEBUG - Transfer packet 21, received 1100/24200/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65651 [9] DEBUG - Transfer packet 22, received 1100/25300/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65651 [9] DEBUG - Transfer packet 23, received 1100/26400/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65651 [9] DEBUG - Transfer packet 24, received 1100/27500/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65651 [9] DEBUG - Transfer packet 25, received 1100/28600/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65651 [9] DEBUG - Transfer packet 26, received 1100/29700/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65651 [9] DEBUG - Transfer packet 27, received 1100/30800/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65651 [9] DEBUG - Transfer packet 28, received 1100/31900/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65791 [3] DEBUG - Transfer packet 29, received 1100/33000/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65792 [7] DEBUG - Transfer packet 30, received 1100/34100/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65792 [8] DEBUG - Transfer packet 31, received 1100/35200/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65792 [9] DEBUG - Transfer packet 32, received 1100/36300/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65792 [3] DEBUG - Transfer packet 33, received 1100/37400/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65792 [7] DEBUG - Transfer packet 34, received 1100/38500/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65793 [8] DEBUG - Transfer packet 35, received 1100/39600/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65933 [8] DEBUG - Transfer packet 36, received 1100/40700/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65933 [8] DEBUG - Transfer packet 37, received 1100/41800/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65934 [3] DEBUG - Transfer packet 38, received 1100/42900/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65934 [7] DEBUG - Transfer packet 39, received 1100/44000/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65935 [9] DEBUG - Transfer packet 40, received 1100/45100/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65935 [8] DEBUG - Transfer packet 41, received 1100/46200/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
65935 [3] DEBUG - Transfer packet 42, received 1100/47300/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66075 [9] DEBUG - Transfer packet 43, received 1100/48400/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66075 [7] DEBUG - Transfer packet 44, received 1100/49500/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66076 [8] DEBUG - Transfer packet 45, received 1100/50600/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66076 [9] DEBUG - Transfer packet 46, received 1100/51700/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66076 [3] DEBUG - Transfer packet 47, received 1100/52800/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66076 [8] DEBUG - Transfer packet 48, received 1100/53900/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66077 [7] DEBUG - Transfer packet 49, received 1100/55000/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66219 [7] DEBUG - Transfer packet 50, received 1100/56100/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66219 [7] DEBUG - Transfer packet 51, received 1100/57200/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66219 [7] DEBUG - Transfer packet 52, received 1100/58300/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66219 [9] DEBUG - Transfer packet 53, received 1100/59400/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66219 [3] DEBUG - Transfer packet 54, received 1100/60500/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66219 [9] DEBUG - Transfer packet 55, received 1100/61600/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66219 [3] DEBUG - Transfer packet 56, received 1100/62700/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66359 [9] DEBUG - Transfer packet 57, received 1100/63800/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66360 [8] DEBUG - Transfer packet 58, received 1100/64900/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66360 [3] DEBUG - Transfer packet 59, received 1100/66000/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66360 [7] DEBUG - Transfer packet 60, received 1100/67100/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66360 [8] DEBUG - Transfer packet 61, received 1100/68200/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66360 [9] DEBUG - Transfer packet 62, received 1100/69300/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66360 [3] DEBUG - Transfer packet 63, received 1100/70400/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66502 [7] DEBUG - Transfer packet 64, received 1100/71500/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66502 [7] DEBUG - Transfer packet 65, received 1100/72600/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66502 [9] DEBUG - Transfer packet 66, received 1100/73700/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66502 [7] DEBUG - Transfer packet 67, received 1100/74800/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66502 [8] DEBUG - Transfer packet 68, received 1100/75900/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66503 [9] DEBUG - Transfer packet 69, received 1100/77000/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66503 [9] DEBUG - Transfer packet 70, received 1100/78100/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66644 [8] DEBUG - Transfer packet 71, received 1100/79200/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66644 [8] DEBUG - Transfer packet 72, received 1100/80300/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66644 [8] DEBUG - Transfer packet 73, received 1100/81400/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66644 [7] DEBUG - Transfer packet 74, received 1100/82500/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66644 [8] DEBUG - Transfer packet 75, received 1100/83600/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66644 [7] DEBUG - Transfer packet 76, received 1100/84700/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66644 [9] DEBUG - Transfer packet 77, received 1100/85800/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66786 [7] DEBUG - Transfer packet 78, received 1100/86900/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66786 [3] DEBUG - Transfer packet 79, received 1100/88000/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66786 [3] DEBUG - Transfer packet 80, received 1100/89100/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66786 [3] DEBUG - Transfer packet 81, received 1100/90200/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66786 [8] DEBUG - Transfer packet 82, received 1100/91300/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66786 [8] DEBUG - Transfer packet 83, received 1100/92400/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66786 [8] DEBUG - Transfer packet 84, received 1100/93500/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66927 [3] DEBUG - Transfer packet 85, received 1100/94600/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66927 [3] DEBUG - Transfer packet 86, received 1100/95700/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66927 [3] DEBUG - Transfer packet 87, received 1100/96800/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66927 [3] DEBUG - Transfer packet 88, received 1100/97900/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02
66928 [3] DEBUG - Transfer packet 89, received 399/98299/98299 bytes for asset b885b231-ad7d-4b42-8fa2-b9cb37c74e02