Author Topic: String Performance: StrRef vs. 2da  (Read 468 times)

Legacy_B_Harrison

  • Sr. Member
  • ****
  • Posts: 301
  • Karma: +0/-0
String Performance: StrRef vs. 2da
« on: August 16, 2010, 11:47:34 pm »


               Is there likely to be any sort of significant performance difference between getting/using string values from a TLK (the various StrRef functions), and doing the same from a 2da (Get2DAString)? I ask, because I'm assuming a few things (perhaps incorrectly):

1) Strings are somewhat slow/large in comparison to other variable types,
2) Retrieving data from a talk table or 2da is faster than creating/storing it within a script.

I have the luxury of writing the game's standard tlk and 2das specifically for a module, so I'd much rather store any appropriate data in those formats than define the many, many strings I'll need inside the module's scripts. Is one better than the other?

Thanks, scripters!
               
               

               


                     Modifié par B_Harrison, 16 août 2010 - 10:48 .
                     
                  


            

Legacy_Lightfoot8

  • Hero Member
  • *****
  • Posts: 4797
  • Karma: +0/-0
String Performance: StrRef vs. 2da
« Reply #1 on: August 17, 2010, 12:02:01 am »


               It may depend on how big your 2da is.
The main difference between the 2da and the .tlk is that the 2da is a sequential file. No line in the 2da has to be the same size. This means that if you want to get line 500 in the 2da all 499 lines before it have to be read to get to line 500.
The .talk file on the other hand has a random access table at the beginning of it. Every entry is at the same size. The table always begins at the same spot in the file. Here when you want to find line 500, It does not have to read the 499 table entries before it. It reads the exact 40 bytes it need for the information it needs about the resref. This 40 bytes contains if there is a string present, where it is located in the file and if there is a sound file that gets played with the string in a conversation. So reads 40 byres. finds where in the file the string is located. then reads just the string out of the file.
In short unless your 2da is real short the tlk is most likely faster.


EDIT:

One thing I did not think about in my answer above is the caching of the 2da's. This could speed up the use of the 2da over and above the tlk especially if you are going to be accessing a lot of lines from the same 2da all at the same time. Since the new 1.69 update I am not really sure what system the game uses to cash the 2da's. I know that the game use to keep the last two 2da's accessed in memory. now that we have the new .ini setting for how many 2da lines are cached. I am not sure if a full 2da is ever kept in memory now.
               
               

               


                     Modifié par Lightfoot8, 17 août 2010 - 12:10 .
                     
                  


            

Legacy_B_Harrison

  • Sr. Member
  • ****
  • Posts: 301
  • Karma: +0/-0
String Performance: StrRef vs. 2da
« Reply #2 on: August 17, 2010, 01:02:31 am »


               Thanks Lightfoot!

That's very clear - most of the 2das I'd be using would be somewhat long (dozens of rows, or more), so I'll make room in the tlk for them instead.
               
               

               


                     Modifié par B_Harrison, 17 août 2010 - 12:02 .
                     
                  


            

Legacy_Lightfoot8

  • Hero Member
  • *****
  • Posts: 4797
  • Karma: +0/-0
String Performance: StrRef vs. 2da
« Reply #3 on: August 17, 2010, 01:11:33 am »


               oops,  Looks like i got my Edit above in a litle late.   lol



I still think the .talk is the best bet.
               
               

               
            

Legacy_Shadooow

  • Hero Member
  • *****
  • Posts: 7698
  • Karma: +0/-0
String Performance: StrRef vs. 2da
« Reply #4 on: August 17, 2010, 05:58:19 pm »


               I would not care about constant strings in script. The script might grow up in compiled size, but I don't think it affect speed, much.



Anyway, TLK must be faster, Lightfoot provided very reasonable explanation.

Also there are other way to do that, local string on some object, name/description of some object etc. Also, if the strings you want to use arent meant for players, you can simply store them inside dialog.tlk on your server only.



Also note, that SendMessageToPCByStrRef should (if lexicon is right) send player message from his own talk table (so server actually sends the line number).
               
               

               
            

Legacy_Lightfoot8

  • Hero Member
  • *****
  • Posts: 4797
  • Karma: +0/-0
String Performance: StrRef vs. 2da
« Reply #5 on: August 17, 2010, 09:32:14 pm »


               Shadow Does make a good point of being able to place the strings on an Object.  If you had all the strings in a 2da, you could cache them to local vars on an object.  First time you try to read one of the string from the object cache the enitre 2da to locals.  Will have a smal proformance hit the during the cache. After that you are just useing up memory with fast access to the strings.
               
               

               
            

Legacy_Guest_invisig0th_*

  • Jr. Member
  • **
  • Posts: 97
  • Karma: +0/-0
String Performance: StrRef vs. 2da
« Reply #6 on: August 18, 2010, 02:19:28 am »


               

Is there likely to be any sort of significant performance difference between getting/using string values from a TLK (the various StrRef functions), and doing the same from a 2da (Get2DAString)?

Assuming that the information above regarding direct access to TLK entries is correct (and it does make sense), there should be no appreciable performance difference between reading in any one typcial 2DA and gettting one entry from the TLK. If you're talking about nanoseconds, there will be differences. If you're talking about milliseconds, then there won't be noticable differences at all.

I ask, because I'm assuming a few things (perhaps incorrectly):

1) Strings are somewhat slow/large in comparison to other variable types,
2) Retrieving data from a talk table or 2da is faster than creating/storing it within a script.

1) Perhaps not as much as you think. Strings are always a little bit more resource intenstive than other simple value types (int, float), but they are not anywhere near as resource intensive as reference types (object). Since in this case I suspect you are not making *changes* to these strings after you read them in, all you are essentially doing there is creating an array of characters, which is pretty much the same thing as an array of ints or floats. An array of floats is indeed slower/larger than a single float value, but not by very much in the larger scheme of things. Same goes for strings.

2) Once you have a script loaded into memory, anything you do in that script that does NOT access the hard drive (i.e. things requiring processor time) is orders of magnitude faster than anything that accesses the hard drive. At least the first time through, retrieving a TLK Entry or loading a 2DA will require hard drive access to get the data, and even the fastest hard drives are pitifully slow due the fact that physical parts need to move to get your data. So almost 100% of your performance hit there will be simply waiting for the hard drive to access and retrieve the data. The actual data retrieved in this case is only a very, very small fraction of your performance hit.

So strictly speaking, if you absolutely wanted to maximize performance, you would indeed keep these values directly in the scripts and reduce the need for extra hard drive access.

In reality, that is not a good approach for two reasons: first, accessing 2DAs and TLK entries is only a drop in the ocean as far as overall game performance. ReAlistically speaking, hitting a few 2DAs or hitting a few TLK entries won't change anything  at all for the end user. For the second reason, see below.

As for concerns about the quantity of the data, there will be no appreciable difference between reading, say, a one line 2DA and a 500 line 2DA. The difference between reading a few hundred bytes versus a few thousand bytes has all but disappeared with modern computers.  A small chunk of data is a small  chunk of data, and a 500 line text file is a very small chunk of data compared to an executable file like the NWN game file.  Unless your 2DA is 100,000 lines long or  something, it just isn't going to be a factor. So the actual size of your 2DA does not really matter here.

Please note that all this applies only the first time you physically read the data from the hard drive. After that, things get much more complicated, when things like caching enter the picture. Loading something into memory the first time is always a horrendously slow process, and manipulation that data in memory afterwards is always lightning fast by comparsion. Parsing the 2DA to find a specific entry would also have a slight cost, but again that happens in memory, so it's nothing compared to the cost of accessing the hard drive.

I have the luxury of writing the game's standard tlk and 2das specifically for a module, so I'd much rather store any appropriate data in those formats than define the many, many strings I'll need inside the module's scripts. Is one better than the other?

The real reason you should not store these strings directly in the scripts is not performance -- it is flexibility and maintainability. If you decide that one of your custom spells should be renamed, it can be a pain to track down references across many scripts and change them. (And a bulk find & replace for a very common term can easily make changes you actually don't want.)  And If someone eventually wants to make a German version of your module, finding and changing strings hard coded in scripts would be a colossal pain. In both cases, updating a TLK or a 2DA is very strightforward and less likely to lead to unintended results.
               
               

               


                     Modifié par invisig0th, 18 août 2010 - 01:22 .