Picking a random bitwise value from a given number

[Legacy_Lightfoot8]

               Double check again please. with x at 0.

32-x = 32
1<<<32 = 0
0 - 1 = -1
nNumber & -1 masks the entire number back.

correct?
 
Dam.   It worked on my first test.   lol.


Guess we are stuck with an if.  

int Shr ( int nNumber, int x)
{
 
 if (x) return (nNumber>>>x) & ( (1<< 32-x )-1);
else return nNumber; 
}

Edit:  moot point with the added if statment.  Your version would be faster 50% of the time.  mine would be faster 3% of the time.    lol,  I think your version wins. 
               
               

               

                     Modifié par Lightfoot8, 16 novembre 2011 - 11:28 .
                     
                  

            

[Legacy_WhiZard]

               You might revise the if (x) to if (x % 32) for those plugging in numbers
outside the range.  Also if you want to cut characters you can change
(1<< 32-x) to (1<<-x)  the right argument is always gotten
mod 32 for all shift operators in NWScript (so in practice it ranges
from 0 to 31 as numbers outside that range will be modularly converted
into it).

EDIT: Oops, accidentally edited out my old post instead of posting a response.  Original text restored (I think).
               
               

               

                     Modifié par WhiZard, 17 novembre 2011 - 10:51 .
                     
                  

            

[Legacy_Lightfoot8]

               

WhiZard wrote...


You might revise the if (x) to if (x % 32) for those plugging in numbers outside the range.  Also if you want to cut characters you can change (1<< 32-x) to (1<<-x)  the right argument is always gotten mod 32 for all shift operators in NWScript (so in practice it ranges from 0 to 31 as numbers outside that range will be modularly converted into it).

I did think about doing a MOD 32,  But had a concern as to wether it was a side effect or a documented result.   The reason for the concern is that it is not NWscript or the NW VM that is doint the MODing.  It is the machine instruction that is doing it.     If it was a side effect (undocumented result)  there could be a differance in how it is handled between platforms.    A little more research, however, has turned up that is is a documented result.   

From the intel instruction set documentation...

The destination operand can be a register or a memory location. The count operand can be an
immediate value or register CL. The count is masked to five bits, which limits the count range
to 0 to 31. A special opcode encoding is provided for a count of 1.

   
 
So yes the( x%32)  or faster (x & 31)  would both work.  

on the ( 1<< -x ) :  Sweet, nice reduction.
               
               

               

                     Modifié par Lightfoot8, 17 novembre 2011 - 03:15 .
                     
                  

            

[Legacy_WhiZard]

               

Lightfoot8 wrote...
Guess we are stuck with an if.  

int Shr ( int nNumber, int x) {return (nNumber>>>x + 2<<~x) & ~(2<<~x);}


Just found a way around the if.
               
               

               

                     Modifié par WhiZard, 17 novembre 2011 - 10:45 .
                     
                  

            

[Legacy_Lightfoot8]

               

WhiZard wrote...

Lightfoot8 wrote...
Guess we are stuck with an if.  

int Shr ( int nNumber, int x) {return (nNumber>>>x + 2<<~x) & ~(2<<~x);}


Just found a way around the if.

Not quite there Wizard.     lets just take the case of  shr(nNumber ,1)  

the  (nNumber>>>x + 2<<~x)  shifts it down  right 3   then  left 31    at this point  the  bit that was in bit positions 4 and 3 of our original number are  now in   bit postions 31 and 30.   They are also all that is left of nNumber. they have also been shifted 27 left.     

the  ~(2<<~x)  then continues to mask out the 31 bit.    leaving us with the bit that was in position  3 shifted left 27 spots to position 30.     with everything else masked out.          I can half see where you are going, I just think you left out part of the equation.          
               
               

               
            

[Legacy_WhiZard]

               

Lightfoot8 wrote...

WhiZard wrote...

Lightfoot8 wrote...
Guess we are stuck with an if.  

int Shr ( int nNumber, int x) {return (nNumber>>>x + 2<<~x) & ~(2<<~x);}


Just found a way around the if.

Not quite there Wizard.     lets just take the case of  shr(nNumber ,1)  

the  (nNumber>>>x + 2<<~x)  shifts it down  right 3        

It should evaluate to

((nNumber>>>x) + (2<<~x)) & ~(2<<~x).

EDIT: Looks like I needed that extra set of parenthesis.
               
               

               

                     Modifié par WhiZard, 18 novembre 2011 - 06:51 .
                     
                  

            

[Legacy_Lightfoot8]

               

WhiZard wrote...

...It should evaluate to

((nNumber>>>x) + (2<<~x)) & ~(2<<~x).

EDIT: Looks like I needed that extra set of parenthesis.

That is very elegant and nicely done.     I worked a a version while at work yesterday,  Your version still shows more style then mine.   Here is mine, If you are interested.

int Shr ( int nNumber, int x)
{
 return  (-((nNumber & 0x80000000) >>> x)) |( (nNumber & 0x7FFFFFFF) >>> x ) ;
}

For some reason I had to load it up with Parentheses, The compiler became quite fickle with precedence
               
               

               
            

[Legacy_Lightfoot8]

               Just a thought.  If someone was running a server and wanted the >>> to operate as an unsinged shift, It would not be that hard to hak the stand alone server and make it one.   The only draw back is any scripts then written using it would be bugged on any servers/single player clients that where not modified.
               
               

               
            

[Legacy_SkywingvL]

               Lightfoot contacted me about this in a PM.

The behavior that the game implements (both NWN1 and NWN2, and likely other deriatives of NWScript) is:

For OP_SHRIGHT (operator>> in NWScript):

if (Left hand expression is negative)
{
Temp = Negate(Left hand expression)
Return (Negate(Temp SIGNED SHIFT RIGHT by Right hand expression low 8 bits))
}
else
{
 Return (Left hand expression SIGNED SHIFT RIGHT by Right hand expression low 8 bits)
}

For OP_USHRIGHT (operator>>>) in NWScript:

Return (Left hand expression SIGNED SHIFT RIGHT by right hand expression low 8 bits)


The reference VM and JIT implementation that I produced for implements unsigned semantics for OP_USHRIGHT. I'll update these to implement the semantics that the standard VM implements, however potentially incorrect they may be.
               
               

               

                     Modifié par SkywingvL, 20 novembre 2011 - 01:37 .
                     
                  

            

[Legacy_SkywingvL]

               

the.gray.fox wrote...

Hello WhiZard,

No offense taken, I perfectly understand your inquiry.
But I have no official documentation to support my claim.

I can only go by logic, seeing how NWscript inarguably replicates a subset of C++ syntax
and operators. The sole extraneous element is the very >>> operator, likely borrowed
from Java (given its popularity) and likely to workaround the absence of unsigned datatypes.
And yes, the >> operator I believe was meant to perform the aritmetic shift.

Now _if_ I am correct, this makes both the >> and the >>> operators bugged.
As much as I like the game, frankly it would not shock me to learn that they are indeed
so, and have never worked as intended from day zero.

Other parts of the language were never fixed in 69 official patches, after all:

-- UDTs are bugged to the point they can trash the compiler if you dare nest them.
-- The switch() construct fails to treat "string literals" as constants.
-- The const keyword is a joke, to the point you better code without it.
-- A constant declared at script level invades the ::scope and takes precedence over any local struct member that happens to possess its same identifier name.
-- The implementation of the float datatype is bugged.
-- The ternary conditional ?: may incorrectly return its false expression if its evaluation directly supplies the argument for a function call (must wrap the whole ?: expression within () to enforce correct token parsing precedence).

There is surely more it is not occurring me at the moment.


-fox

Could you clarify what the perceived problem with the float data type is (and what the problem with the const qualifier is)?

There are known issues with nested structs in the standard compiler.  The Advanced Script Compiler (http://nwvault.ign.c...ns.Detail&id=99 -- the standalone EXE version fully supports NWN1, or you can use virusman's toolset plugin) will throw a compile time error in compatibility mode if you use structs in such a way that will break the standard compiler.  In extensions mode (-e), you can nest structs as 'expected'.

The scoping of symbols is likely something I would consider by design.
The limiting of switch case blocks to int data types is something I would consider by design.
If you have a specific test case for a lexing problem with operator?:, I may be able to take a look at what's going on there (though it has been a long time since I have looked at the lexer logic).

If you are aware of additional issues, I am always interested to know what they are.  The Advanced Script Compiler contains a number of fixes for many behaviors that are not 'expected' when operated in extensions mode.
               
               

               
            

[Legacy_WhiZard]

               Should the NWN Lexicon also be contacted?  Looking at their online announcement at Palmergames they state:

      

No, the GLLUB team does not plan any further updates.  This is it.

               
               

               
            

[Legacy_SkywingvL]

               

WhiZard wrote...

FunkySwerve wrote...
It smacks of conspiracy theorizing. '>

Theorizing, yes.  Conspiracy, well I'll just repeat '>.

I am not saying all bit shifts just the bit shifts performed on characters.  For example ALT + 0255 character might left shift to a ALT + 0254 character.

Still not really understanding what you're saying in the slightest. Earlier you said:

Given the lyceum examples are all for one byte, it could be that what is being reported for >> and >>> could be for "char" and not "int".

I don't see anything about bit shifts on characters there, or anywhere else in this thread. You've totally lost me. If bit shifts were somehow casting their results into an 8-bit datatype at any stage, you couldn't do what I did in the function on the previous page - data loss would produce wildly distorted results. I really have no idea what you're suggesting anymore. Is anyone else able to understand what WhiZard is suggesting?

Funky

I am saying there might be two different shifts for each bit shift symbol (>>, >>>, <<) one that will work for characters and one that will work for integers.  The result for integers may have been an afterthought, with the intended behavior provided for the use on characters.  Hope that helps.  And, yes, I am in far off speculation stage.

To clear things up a little bit -

There are several fundamental data types supported by the script environment at the lowest levels (runtime and compiler).  They are:

- A signed 32-bit integer ('int' in the script compiler).
- A 32-bit IEEE float ('float' in the script compiler).
- A character string ('string' in the script compiler).  String typed values can only be initialized to a default value, string literal, concatenated (operator +), copied, or compared for equality with other string typed values.
- A void type for declaring functions with no return value only ('void' in the script compiler).
- A 32-bit object id ('object') in the script compiler, for which two manifest constants (OBJECT_INVALID and OBJECT_SELF) exist in the runtime and the compiler, plus whatever other arbitrary values the script host program defines.  Object typed values can only be initialized to a default value (or two manifest constant values), copied, and compared for equality with other values of type object.
- The 'vector' type, which is internally treated as a structure aggregating three floats.  The runtime is only minimally aware of 'vector' in the sense that there is an instruction type to operate on groups of three floats on the stack.  The compiler treats the 'vector' type as a predefined struct with three float members (x, y, z), and some additional initializer syntax support and operator support (generally the same operators afforded to 'float' types).
- Various 'engine structure' types, such as 'talent', 'effect', and so forth.  These are opaque handles to data structures defined by the script host program and can only be initialized to a default value, copied, and compared for equality with engine structure types of the same family.
- The 'action' type, which is not really a data type per se, but an indicator to the compiler that it must emit a special instruction causing the runtime to fork off a copy of the script program's state for deferred execution (i.e. the creation of a 'script situation').  The 'action' pseudo-type cannot be used outside of the declaration of an engine function which may only be performed in nwscript.nss.

Structures can be aggregates of all of the above except 'void' and 'action'; structures are always passed by value and are logically just shorthand for copying multiple variables at once.  The standard compiler does not correctly support nested structures.  The runtime is only minimally aware of structures in that there is an instruction to copy multiple values on the stack, and an instruction to compare multiple values on the stack.

There is no 8-bit integral type implemented natively.

The standard compiler does not support any C preprocessor constructs and has no notion of pragmas or similar directives.

The Advanced Script Compiler (in extensions mode or when used with virusman's toolset plugin) supports a limited preprocessor language that is a subset of the C preprocessor language, including a pragma directive and very basic ifdef support.  Additionally, in extensions mode, nested structures are supported correctly.


On top of the support outlined above, a script host program (such as NWN1 or NWN2, or other NWScript-derived games) can provide additional services in the form of engine functions - for example, engine functions to return the length of a string type, to create a substring derived from an existing string type, to print a string to a log file, etc.  The compiler and runtime environment are structured to be agnostic of the nature of these additional 'engine function' services, other than providing a generic mechanism to allow extended functionality to be implemented by the script host program and then called upon by script.

I do feel compelled to point out that for all of the features that one would have liked NWScript to have, the design was well architected on the whole.  Remember that at the time, there was no CLR or .NET easily accessible for embedding - compared to previous games with cumbersome and highly implementation specific scripting systems (or source-text based interpreters), NWScript was a significant leap forward - the actual script compiler and execution environment are well isolated from the implementation details of the game in a way that makes it easily portable to other games, and a number of powerful constructs are exposed to script in relatively simple and elegant ways (such as deferring execution with 'action' functions).

While the implementation has left things to be desired at times, the number of different games that have used NWScript in a way that have essentially left the core compiler and runtime environment completely untouched are a solid testament to the design forethought put in to the scripting system.
               
               

               

                     Modifié par SkywingvL, 20 novembre 2011 - 06:58 .
                     
                  

            

[Legacy_Lightfoot8]

               

SkywingvL wrote...

the.gray.fox wrote...

;;;

-- The implementation of the float datatype is bugged.
...

There is surely more it is not occurring me at the moment.


-fox

Could you clarify what the perceived problem with the float data type is (and what the problem with the const qualifier is)?

...

I look at this a little last night.  I found the results odd to say the least.   I am currently short on time, So I will just give a short recap of my findings. 

When running the script. 


void main()
{
  float  y = 1.0;
  float z = 3.14;
  float  a= z/y;
  SendMessageToPC(GetFirstPC(), FloatToString(a));
}

The value of  'a' comes out to be.    3.139999866.

When the constant 3.14 is originaly pushed onto the stack everything goes correctly.   With the constant being loaded into the the FPU. (extending it to an 80 bit float)   from the stack( ss: segment)  and then stored onto the VM-stack( heap,  DS: segment. )   

When the float is loaded into FPU for the VM DIVFF instruction ( ByteCode 0x17,0x21),   It is loaded directly into the FPU from the VM-Stack (DS:segment).   At this point it is incorrectly extended to the 80 bit float.   To me this looks more like a bug in the FPU related to moving 32 bit floats into the FPU from the DS:segment.     

I would love to know if everyone gets the same result or not,  Or if other FPU's do not suffer from this bug.  
               
               

               
            

[Legacy_SkywingvL]

               Which segments are involved here are immaterial to this problem.

Any time you use an IEEE floating point value, of which the NWScript 'float' and 'vector' types use, you must be prepared to deal with the fact that IEEE floats are approximations only and not exact values. Note that you need to be prepared to handle different results returned by NWScript for floating point values within generally acceptable margins of error on different platforms.

For example, my NWScript VM uses SSE2 for DIVFF and yields 3.140000105. This is similar to what the NWN2 client (nwn2main.exe), which is built for SSE2, would yield - unless one is running on an old AMD processor when the non-SSE2 version is used, in which case higher precision (80 bits) would be used for intermediate calculations due to x87 FPU usage.

Similarly, the NWN2 server would yield results as though x87 instructions instead of SSE2 instructions were used as it's built without SSE2 usage.

I suspect NWN1 always uses x87 instructions given it's age. Even discounting differences between SSE2 and x87 floating point modes, however, floating point values are fundamentally approximations in any language using IEEE floats. For reliable operation, one always has to be prepared to deal with (and when necessary work to minimize) rounding errors like this.

Note that NWScript floats are always spilled to a 32-bit representation after every operation.  Thus, they will have slightly increased levels of rounding imprecision than a comparable sequence of operations in C on native x87 that might be more likely to remain entirely in the higher precision FPU registers throughout more of the calculations.

I will also add that the usage of 'const' may also introduce differences in the exact rounding errors you see.  Some NWScript compilers may collapse certain 'const' expressions (interpreting them at compile time), potentially with slightly different levels of precision than were they implemented in 'native' NWScript.

The take away here is that there are a number of scenarios that will result in small imprecision errors creeping in to floating point calculations.  Take appropriate steps to deal with these errors (such as using an appropriate FLT_EPSILON value when dealing with floating point comparisons).  Remember that as you perform increasing numbers of floating point operations on a particular value, greater levels of cumulative precision errors will be incurred.
               
               

               

                     Modifié par SkywingvL, 20 novembre 2011 - 07:24 .
                     
                  

            

[Legacy_Lightfoot8]

               There is no way to argue about the IEE standard being  not exact values. That is given in the standard but is not what the complaint from the.gray.fox was about.  It was about the precision of the numbers.   The 3.140000105 that your VM yeilded is well within the 6.5 digit precision that the 32 bit float is documented to.    And yes with multiple computations errors will creap in and erode that presision.   your number even exceeds the 6.5 digits and is giving a full 7 digits of precision.   The 3.13999998 on the other hand is only giving  2 digits of percision.  With only a single divison by one.    If that can not be concidered a bug I do not know what can.    I am not even saying that one of the numbers is more inaccurate then the other one is.   I am just saying that the percision  is out of wack. 

as far as the segmentation regesters not being the problem you are correct, they are not.     I was tired when I was looking at it and misinterpted what I was looking at.    The problem does not seem to be in the VM at all.  It looks to be in the standard compiler.   

it compiles  float int x =3.14 to 
..
02 04   ..............    RSADDF
04 04 4048f5C2  CONSTF 3.1400000
....

The above is what nwExplorer shows. 

The problem is that    4048f5C2..  is not 3.14  it is the 3.13999886648  that we are getting as our result with lack of percision.    It should have been compiled more to 4048F5C3 that is 3.140000104   

I have not looked at the compiler much.   But the problem as reported by The Gray Fox is the constants being feed to the VM from the compiler.   

A new test script would be

void main()
{
  float x = 3.14;
  SendMessageToPC(GetFirstPC(), FloatToString(x));
}

It still gives the 3.13999886648 result, because that is what was given to it by the compiler.

EDIT: This is all from the standard script compiler, not any of the extended/advanced ones.  I have not looked at any of them.
               
               

               

                     Modifié par Lightfoot8, 21 novembre 2011 - 12:50 .