Changing IRoutineMetadata in Plugin::makeProcedure

classic Classic list List threaded Threaded
32 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Changing IRoutineMetadata in Plugin::makeProcedure

Jiří Činčura-2
Hi *,

in Plugin::makeProcedure I'm getting IRoutineMetadata and input and
output builder. Can I use these, somehow, to change what values the
server is going to send me later? I.e. calling getInputMetadata,
changing types (in my case currently from SQL_TYPE_DATE to
SQL_TIMESTAMP), then using builder->getMetadata and using this new
metadata later in my IExternalProcedureImpl? In other words, is then the
data passed in "void* msg" i.e. to Procedure::open going to be in new
format?

--
Mgr. Jiří Činčura
https://www.tabsoverspaces.com/

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Adriano dos Santos Fernandes-3
On 18/04/2017 11:15, Jiří Činčura wrote:

> Hi *,
>
> in Plugin::makeProcedure I'm getting IRoutineMetadata and input and
> output builder. Can I use these, somehow, to change what values the
> server is going to send me later? I.e. calling getInputMetadata,
> changing types (in my case currently from SQL_TYPE_DATE to
> SQL_TIMESTAMP), then using builder->getMetadata and using this new
> metadata later in my IExternalProcedureImpl? In other words, is then the
> data passed in "void* msg" i.e. to Procedure::open going to be in new
> format?
>
Yes, sure.


Adriano


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Jiří Činčura-2
> Yes, sure.

Should changing the type be enough? Because I changed it from
SQL_TYPE_DATE to SQL_TIMESTAMP, but I'm still getting length == 4, while
the timestamp should be 8, isn't it?

--
Mgr. Jiří Činčura
https://www.tabsoverspaces.com/

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Adriano dos Santos Fernandes-3
On 18/04/2017 11:27, Jiří Činčura wrote:
>> Yes, sure.
> Should changing the type be enough? Because I changed it from
> SQL_TYPE_DATE to SQL_TIMESTAMP, but I'm still getting length == 4, while
> the timestamp should be 8, isn't it?
>
You should call setLength and setScale (if necessary) too.


Adriano


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Jiří Činčura-2
> You should call setLength and setScale (if necessary) too.

Ah, that's what I was missing. Isn't there a constant for i.e.
SQL_TIMESTAMP? Because it's always 8 bytes, but feels wrong to hardcode
it manually myself.

Also going from SQL_TEXT to SQL_VARYING should be possible, right? How
I'm going to handle the length here. I don't know the real length of the
string, do I?

--
Mgr. Jiří Činčura
https://www.tabsoverspaces.com/

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Dimitry Sibiryakov-3
18.04.2017 16:40, Jiří Činčura wrote:
> Isn't there a constant for i.e.
> SQL_TIMESTAMP? Because it's always 8 bytes, but feels wrong to hardcode
> it manually myself.

   sizeof(ISC_TIMESTAMP)


--
   WBR, SD.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Adriano dos Santos Fernandes-3
In reply to this post by Jiří Činčura-2
On 18/04/2017 11:40, Jiří Činčura wrote:
>> You should call setLength and setScale (if necessary) too.
> Ah, that's what I was missing. Isn't there a constant for i.e.
> SQL_TIMESTAMP? Because it's always 8 bytes, but feels wrong to hardcode
> it manually myself.

There is no constants, in C/C++ some sizeof may be possible.

The data passed to these methods are still old kind of data not mapped
to the new APIs.


> Also going from SQL_TEXT to SQL_VARYING should be possible, right? How
> I'm going to handle the length here. I don't know the real length of the
> string, do I?
>
I always do it.

See fixMetadata here in the Node.js driver.

https://github.com/asfernandes/node-firebird-driver-native/blob/master/src/lib/fb-util.ts


Adriano


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Dmitry Yemanov-3
In reply to this post by Jiří Činčura-2
18.04.2017 17:40, Jiří Činčura wrote:
>
> Also going from SQL_TEXT to SQL_VARYING should be possible, right?

Generally it is, unless SQL_TEXT is of length 32766 or 32767.


Dmitry


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Dimitry Sibiryakov-3
18.04.2017 16:52, Dmitry Yemanov wrote:
> Generally it is, unless SQL_TEXT is of length 32766 or 32767.

   Isn't vary.vary_length declared as unsigned short?


--
   WBR, SD.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Jiří Činčura-2
In reply to this post by Adriano dos Santos Fernandes-3
> See fixMetadata here in the Node.js driver.
>
> https://github.com/asfernandes/node-firebird-driver-native/blob/master/src/lib/fb-util.ts

That's what I do exactly. But then I'm getting in length* full length of
the field (as if it's CHAR) and not real length (as in VARCHAR). So for
CHAR(20) UTF8, changed to SQL_VARYING, the length is 80. Which is kind
of confusing. :)

* You're reading it here:
https://github.com/asfernandes/node-firebird-driver-native/blob/master/src/lib/fb-util.ts#L135
.

--
Mgr. Jiří Činčura
https://www.tabsoverspaces.com/

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Dimitry Sibiryakov-3
18.04.2017 17:03, Jiří Činčura wrote:
> So for CHAR(20) UTF8

   CHAR and UTF-8 (generally, any variable character length charset) must never be used
together.


--
   WBR, SD.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Jiří Činčura-2
>    CHAR and UTF-8 (generally, any variable character length charset) must
>    never be used
> together.

Hmm. Can you elaborate on that?

Server allows that. And if developer creates such .NET UDR, I have to
somehow handle it. Of course I can declare CHAR not supported at all,
but that's kind of lame.

--
Mgr. Jiří Činčura
https://www.tabsoverspaces.com/

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Dimitry Sibiryakov-3
18.04.2017 17:13, Jiří Činčura wrote:
> Hmm. Can you elaborate on that?

   Server doesn't determine real length of string in characters, so it just pad string up
to 4*MaxChars bytes with spaces. That's why for CHAR(20) you always get 80 bytes of data,
which can contain more that 20 characters if you count trailing spaces as well.
   And there is no place in API where real length in bytes can be provided for CHAR() type.
   Because of that handling of CHAR() is broken in many places for UTF-8 and alike charsets.

--
   WBR, SD.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Mark Rotteveel-2
On 18-4-2017 17:20, Dimitry Sibiryakov wrote:
> 18.04.2017 17:13, Jiří Činčura wrote:
>> Hmm. Can you elaborate on that?
>
>    Server doesn't determine real length of string in characters, so it just pad string up
> to 4*MaxChars bytes with spaces. That's why for CHAR(20) you always get 80 bytes of data,
> which can contain more that 20 characters if you count trailing spaces as well.
>    And there is no place in API where real length in bytes can be provided for CHAR() type.
>    Because of that handling of CHAR() is broken in many places for UTF-8 and alike charsets.
>

Real length in characters of a CHAR is length / max bytes per character,
so for UTF-8: length / 4.

Mark
--
Mark Rotteveel

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Dimitry Sibiryakov-3
18.04.2017 17:23, Mark Rotteveel wrote:
> Real length in characters of a CHAR is length / max bytes per character,
> so for UTF-8: length / 4.

   But if you count real character in this buffer, you can get up to 80 of them.


--
   WBR, SD.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Jiří Činčura-2
In reply to this post by Dimitry Sibiryakov-3
Hmm. That explains a lot of issues I was trying to push through today with CHARs. I.e. was getting 80 bytes buffer where was 20 characters and 60 spaces, because the characters were all 1B.

It's pretty sad, actually. 

So my best chance is probably to trim to length/4 and hope for the best. 

--
Mgr. Jiří Činčura
https://www.tabsoverspaces.com/



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Jiří Činčura-2
It's always going to be spaces/0s. I don't see a way for otherwise. 

--
Mgr. Jiří Činčura
https://www.tabsoverspaces.com/


On Tue, Apr 18, 2017, at 17:43, Dimitry Sibiryakov wrote:
> 18.04.2017 17:38, Jiří Činčura wrote:
> > So my best chance is probably to trim to length/4 and hope for the best.
>
>    Trim only spaces (zeros for OCTETS).
>
>
> --
>    WBR, SD.


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Adriano dos Santos Fernandes-3
In reply to this post by Jiří Činčura-2
On 18/04/2017 12:38, Jiří Činčura wrote:
> Hmm. That explains a lot of issues I was trying to push through today
> with CHARs. I.e. was getting 80 bytes buffer where was 20 characters
> and 60 spaces, because the characters were all 1B.
>
> It's pretty sad, actually.
>
> So my best chance is probably to trim to length/4 and hope for the best.
>
>
Jiri, what you said (change SQL_TEXT to SQL_VARYING) and what I show you
(fixMetadata function) just fix this problem.

Change it and forget. You'll always read SQL_VARYING and have the actual
length in the first two bytes.

The problem is not with CHAR data type, only its representation with
SQL_TEXT.

This change from SQL_TEXT->SQL_VARYING will not work only in some old
versions (maybe firsts 2.1.x or 2.0.x).


Adriano


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Jiří Činčura-2
> Jiri, what you said (change SQL_TEXT to SQL_VARYING) and what I show you
> (fixMetadata function) just fix this problem.

I wish. See below.

Also I have to investigate the 32766 or 32767 limit Dmitry Y. mentioned.
I suppose he's not lying. :) That would make it suck even more.

> Change it and forget. You'll always read SQL_VARYING and have the actual
> length in the first two bytes.

This is what I'm getting from the server. The field is CHAR(20) UTF8.
The length from metadata is 80. The memory dump where the pointer points
is:
{ 80, 0, 49, 50, 51, 52, 53, 54, 55, 56, 57, 48, 49, 50, 51, 52, 53, 54,
55, 56, 57, 48, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32,
32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32,
32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32,
32, 32, 32, 32, 32, 32, 32, 32, 32 }

Which clearly shows the first two bytes are reporting 80 as well. Fair
enough. Reading that buffer as UTF8 (or just looking at it, is basically
plain US-ASCII, nothing fancy) you get
"12345678901234567890____________________________________________________________"
(I replaced the spaces with visible character). It's clearly correct
string at the beginning and then 60(!) spaces. And that doesn't look
correct to me at all.

I think that's what Dimitry S. was pointing to.

With all that, unless I'm doing something wrong, I'd need to do some
trimming (or forgot about CHARs at all). The conversion to SQL_VARYING
gets me only so far.

> This change from SQL_TEXT->SQL_VARYING will not work only in some old
> versions (maybe firsts 2.1.x or 2.0.x).

Fine for me, as external engine needs 3.0 anyway.

--
Mgr. Jiří Činčura
https://www.tabsoverspaces.com/

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Changing IRoutineMetadata in Plugin::makeProcedure

Slavomir Skopalik
Hi Jiri,
you must decode input UTF8 buffer into string(probably unicode) and next
you have to trim trailing spaces.
For text, just put some non US characters into text.

Slavek

> The length from metadata is 80. The memory dump where the pointer points
> is:
> { 80, 0, 49, 50, 51, 52, 53, 54, 55, 56, 57, 48, 49, 50, 51, 52, 53, 54,
> 55, 56, 57, 48, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32,
> 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32,
> 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32,
> 32, 32, 32, 32, 32, 32, 32, 32, 32 }
>
> Which clearly shows the first two bytes are reporting 80 as well. Fair
> enough. Reading that buffer as UTF8 (or just looking at it, is basically
> plain US-ASCII, nothing fancy) you get
> "12345678901234567890____________________________________________________________"
> (I replaced the spaces with visible character). It's clearly correct
> string at the beginning and then 60(!) spaces. And that doesn't look
> correct to me at all.
>
> I think that's what Dimitry S. was pointing to.
>
> With all that, unless I'm doing something wrong, I'd need to do some
> trimming (or forgot about CHARs at all). The conversion to SQL_VARYING
> gets me only so far.
>
>> This change from SQL_TEXT->SQL_VARYING will not work only in some old
>> versions (maybe firsts 2.1.x or 2.0.x).
> Fine for me, as external engine needs 3.0 anyway.
>



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
12
Loading...