Scale of number and precision on floating data types

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Scale of number and precision on floating data types

Köditz, Martin

Hi all,

 

I hang on a text part in fblangref25-datatypes.xml -> fblangref25-datatypes-floattypes:

Floating-point data types are examples of data stored in the DBMS with the precision matching the scale of the number.

 

What is the scale of number? Please make this clear. Otherwise I don’t get the correct translation.

 

Regards

Martin


------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Firebird-docs mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/firebird-docs
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Scale of number and precision on floating data types

firebirdlists
Hello ,

Wednesday, August 10, 2016, 9:20:50 PM, Martin Koeditz wrote:

> I hang on a text part in fblangref25-datatypes.xml -> fblangref25-datatypes-floattypes:
> Floating-point data types are examples of data stored in the DBMS with the precision matching the scale of the number.
 
> What is the scale of number? Please make this clear. Otherwise I don’t get the correct translation.
 
I don't like this at all.  I think it needs some input from Paul and
Dmitry and we should rewrite it.

The way I understand the storage of floating point numbers:

FLOAT is stored as INTEGER (precision 9, scale 0) with the scale
(stored separately) depending on the position of the decimal point.

DOUBLE PRECISION is stored as BIGINT (18,0) with scale
depending on the position of the decimal point.

So (as I understand), floating point and fixed point numbers are
stored in the same manner.  The difference - understood by the engine
- is that the *accuracy* of floating point numbers is reduced as the
scale increases.  The *accuracy* of the last two digits is always
unreliable;  hence, the more the decimal point moves to the right, the
less accurate the number.

This assumption does need confirmation or correction, though.

The correctness of the statement aside, is your question about how to
translate the words "precision" and "scale"?  Possibly Thomas can help
with that, as he did the German translations of quite a few of the
earlier documents and often writes articles for German publications.

Helen


------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Firebird-docs mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/firebird-docs
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Scale of number and precision on floating data types

Dmitry Yemanov-3
10.08.2016 23:35, Helen Borrie wrote:
>
>> I hang on a text part in fblangref25-datatypes.xml -> fblangref25-datatypes-floattypes:
>> Floating-point data types are examples of data stored in the DBMS with the precision matching the scale of the number.
>
>> What is the scale of number? Please make this clear. Otherwise I don’t get the correct translation.
>
> I don't like this at all.  I think it needs some input from Paul and
> Dmitry and we should rewrite it.

I agree the wording is incorrect. The Russian original is slightly
better but still far from being a good one.

> The way I understand the storage of floating point numbers:
>
> FLOAT is stored as INTEGER (precision 9, scale 0) with the scale
> (stored separately) depending on the position of the decimal point.
>
> DOUBLE PRECISION is stored as BIGINT (18,0) with scale
> depending on the position of the decimal point.

Nope, floating point numbers are stored completely different to the
fixed point numbers. FLOAT is single-precision format which occupies 4
bytes. DOUBLE is double-precision format which occupies 8 bytes. But
they have no relationship to INT/BIGINT despite the byte size (4/8
bytes), the binary layout is different (see IEEE 754) and it embeds
{sign, exponent, mantissa}.

I suppose the intended meaning was something like: floating point data
types represent data stored in the database with the dynamic precision
corresponding its storage format (physical size of the value).


Dmitry


------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Firebird-docs mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/firebird-docs
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Scale of number and precision on floating data types

firebirdlists
Hello Dmitry,

Thursday, August 11, 2016, 9:42:48 AM, you wrote:

> Nope, floating point numbers are stored completely different to the
> fixed point numbers. FLOAT is single-precision format which occupies 4
> bytes. DOUBLE is double-precision format which occupies 8 bytes. But
> they have no relationship to INT/BIGINT despite the byte size (4/8
> bytes), the binary layout is different (see IEEE 754) and it embeds
> {sign, exponent, mantissa}.

> I suppose the intended meaning was something like: floating point data
> types represent data stored in the database with the dynamic precision
> corresponding its storage format (physical size of the value).

OK...how about this wording to substitute for the troublesome
translation:
"
Floating point data types are stored in an IEEE 754 binary format that
comprises sign, exponent and mantissa.  Precision is dynamic,
corresponding to the physical storage format of the value, which may
be up to 4 bytes for the FLOAT type and up to 8 bytes for DOUBLE
PRECISION.
"

I would be loath to mention "scale" at all as it is not relevant for
floating-point types.

Comments?  Enhancements?  Martin, if that wording between the quotes
is a correct interpretation, would it work for your translation?

Helen



------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Firebird-docs mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/firebird-docs
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Scale of number and precision on floating data types

Dmitry Yemanov-3
11.08.2016 06:38, Helen Borrie wrote:
>
> Floating point data types are stored in an IEEE 754 binary format that
> comprises sign, exponent and mantissa.  Precision is dynamic,
> corresponding to the physical storage format of the value, which may
> be up to 4 bytes for the FLOAT type and up to 8 bytes for DOUBLE
> PRECISION.

It's exactly either 4 or 8 bytes, not "up to".


Dmitry



------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Firebird-docs mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/firebird-docs
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Scale of number and precision on floating data types

Köditz, Martin
The wording is OK. I will use it.

Martin

-----Ursprüngliche Nachricht-----
Von: Dmitry Yemanov [mailto:[hidden email]]
Gesendet: Donnerstag, 11. August 2016 07:19
An: Chatter regarding Firebird documentation
Betreff: Re: [Firebird-docs] Scale of number and precision on floating data types

11.08.2016 06:38, Helen Borrie wrote:
>
> Floating point data types are stored in an IEEE 754 binary format that
> comprises sign, exponent and mantissa.  Precision is dynamic,
> corresponding to the physical storage format of the value, which may
> be up to 4 bytes for the FLOAT type and up to 8 bytes for DOUBLE
> PRECISION.

It's exactly either 4 or 8 bytes, not "up to".


Dmitry



------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports. http://sdm.link/zohodev2dev _______________________________________________
Firebird-docs mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/firebird-docs

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Firebird-docs mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/firebird-docs
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Scale of number and precision on floating data types

firebirdlists
Hello Martin,

Thursday, August 11, 2016, 6:00:52 PM, you wrote:

> The wording is OK. I will use it.

Did you catch Dmitry's correction?


> -----Ursprüngliche Nachricht-----
> Von: Dmitry Yemanov [mailto:[hidden email]]
> Gesendet: Donnerstag, 11. August 2016 07:19
> An: Chatter regarding Firebird documentation
> Betreff: Re: [Firebird-docs] Scale of number and precision on floating data types

> 11.08.2016 06:38, Helen Borrie wrote:
>>
>> Floating point data types are stored in an IEEE 754 binary format that
>> comprises sign, exponent and mantissa.  Precision is dynamic,
>> corresponding to the physical storage format of the value, which may
>> be up to 4 bytes for the FLOAT type and up to 8 bytes for DOUBLE
>> PRECISION.

> It's exactly either 4 or 8 bytes, not "up to".


> Dmitry



> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth
> and traffic patterns at an interface-level. Reveals which users,
> apps, and protocols are consuming the most bandwidth. Provides
> multi-vendor support for NetFlow, J-Flow, sFlow and other flows.
> Make informed decisions using capacity planning reports.
> http://sdm.link/zohodev2dev
> _______________________________________________
> Firebird-docs mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/firebird-docs

> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. http://sdm.link/zohodev2dev
> _______________________________________________
> Firebird-docs mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/firebird-docs



--
Kind regards,
 Helen Borrie


------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Firebird-docs mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/firebird-docs
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Scale of number and precision on floating data types

Köditz, Martin
Hi Helen,

yes i did.

Martin

-----Ursprüngliche Nachricht-----
Von: Helen Borrie [mailto:[hidden email]]
Gesendet: Donnerstag, 11. August 2016 09:26
An: Chatter regarding Firebird documentation
Betreff: Re: [Firebird-docs] Scale of number and precision on floating data types

Hello Martin,

Thursday, August 11, 2016, 6:00:52 PM, you wrote:

> The wording is OK. I will use it.

Did you catch Dmitry's correction?


> -----Ursprüngliche Nachricht-----
> Von: Dmitry Yemanov [mailto:[hidden email]]
> Gesendet: Donnerstag, 11. August 2016 07:19
> An: Chatter regarding Firebird documentation
> Betreff: Re: [Firebird-docs] Scale of number and precision on floating
> data types

> 11.08.2016 06:38, Helen Borrie wrote:
>>
>> Floating point data types are stored in an IEEE 754 binary format
>> that comprises sign, exponent and mantissa.  Precision is dynamic,
>> corresponding to the physical storage format of the value, which may
>> be up to 4 bytes for the FLOAT type and up to 8 bytes for DOUBLE
>> PRECISION.

> It's exactly either 4 or 8 bytes, not "up to".


> Dmitry



> ----------------------------------------------------------------------
> -------- What NetFlow Analyzer can do for you? Monitors network
> bandwidth and traffic patterns at an interface-level. Reveals which
> users, apps, and protocols are consuming the most bandwidth. Provides
> multi-vendor support for NetFlow, J-Flow, sFlow and other flows.
> Make informed decisions using capacity planning reports.
> http://sdm.link/zohodev2dev
> _______________________________________________
> Firebird-docs mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/firebird-docs

> ----------------------------------------------------------------------
> -------- What NetFlow Analyzer can do for you? Monitors network
> bandwidth and traffic patterns at an interface-level. Reveals which
> users, apps, and protocols are consuming the most bandwidth. Provides
> multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make
> informed decisions using capacity planning reports.
> http://sdm.link/zohodev2dev 
> _______________________________________________
> Firebird-docs mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/firebird-docs



--
Kind regards,
 Helen Borrie


------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Firebird-docs mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/firebird-docs

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Firebird-docs mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/firebird-docs
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Scale of number and precision on floating data types

Paul Vinkenoog
In reply to this post by Dmitry Yemanov-3
Hi all,

>> Floating point data types are stored in an IEEE 754 binary format that
>> comprises sign, exponent and mantissa.  Precision is dynamic,
>> corresponding to the physical storage format of the value, which may
>> be up to 4 bytes for the FLOAT type and up to 8 bytes for DOUBLE
>> PRECISION.
>
> It's exactly either 4 or 8 bytes, not "up to".

The storage used is 4 or 8 bytes, but the precision is less, because a
number of bits are used for sign and exponent.

FLOAT precision is 24 bits - roughly 7 decimal digits.
The binary exponent's range is -126..127, which - again, roughly -
corresponds to -38..38 base 10.

DOUBLE precision is 53 bits (around 16 decimal digits) and the exponent
ranges from -1022 to 1023 binary or roughly -308 to 308 decimal.

The precision is not dynamic (except when precision is lost because the
number is too small to be normalized, but I think that's outside the
scope of an SQL reference).

Cheers,
Paul


------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Firebird-docs mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/firebird-docs
Loading...