POST and UTF-8

This is the forum for miscellaneous technical/programming questions.

Moderator: 2ffat

POST and UTF-8

Postby Lena » Sun May 28, 2017 3:45 am

Hi.
I saved all *.php file in UTF-8.
What is the correct way to see Russian letters?

Code: Select all
//Windows 7
std::unique_ptr<THTTPClient> aHTTP(THTTPClient::Create());
   std::unique_ptr<TStringList> paramlist(new TStringList());
   std::unique_ptr<TStringStream> Response(new TStringStream());
   String path = GetCurrentDir() + L"\\link.ini";
   std::unique_ptr<TIniFile> FileINI(new TIniFile(path));
   //http://mysite/pushTest/api.php
   String postlink = FileINI->ReadString(L"LINK", L"post", "");

   String postmetod = L"method=sendPush";
   String postheader = L"title=" + Trim(Edit1->Text);
   String postbody = L"text=" + Trim(Edit2->Text);
   paramlist->Add(postmetod);
   paramlist->Add(postheader);
   paramlist->Add(postbody);
   aHTTP->Post(postlink,paramlist.get(),Response.get());
   rezult = Response->DataString;
   ShowMessage(rezult);//I do not see Russian letters
Lena
BCBJ Master
BCBJ Master
 
Posts: 480
Joined: Sun Feb 06, 2011 1:28 pm

Re: POST and UTF-8

Postby Lena » Tue May 30, 2017 12:18 am

It looks like this:
Code: Select all
std::unique_ptr<TStringStream> Response(new TStringStream("",TEncoding::UTF8, false));

Why after changing the encoding I can not parse JSON?
Code: Select all
String rezult = "";
   std::unique_ptr<THTTPClient> aHTTP(THTTPClient::Create());
   std::unique_ptr<TStringList> paramlist(new TStringList());
   std::unique_ptr<TStringStream> Response(new TStringStream("",TEncoding::UTF8, false));

   String path = GetCurrentDir() + L"\\link.ini";
   std::unique_ptr<TIniFile> FileINI(new TIniFile(path));
   //http://mysite/pushTest/api.php
   String postlink = FileINI->ReadString(L"LINK", L"post", "");

   String postmetod = L"method=sendPush1";
   String postheader = L"title=" + Trim(Edit1->Text);
   String postbody = L"text=" + Trim(Edit2->Text);
   paramlist->Add(postmetod);
   paramlist->Add(postheader);
   paramlist->Add(postbody);
   aHTTP->Post(postlink,paramlist.get(),Response.get());
   rezult = Response->DataString;

if(rezult != "")
   {
   ShowMessage(rezult);
   std::unique_ptr<TJSONValue> LJSONValue(TJSONObject::ParseJSONValue(rezult));
   TJSONObject *LJSONObject = dynamic_cast<TJSONObject*>(LJSONValue.get());
   if(LJSONObject != NULL)//problem always NULL
    {
          //
         }
Attachments
err.jpg
err.jpg (18.77 KiB) Viewed 2368 times
Lena
BCBJ Master
BCBJ Master
 
Posts: 480
Joined: Sun Feb 06, 2011 1:28 pm

Re: POST and UTF-8

Postby Lena » Tue May 30, 2017 3:36 am

Fix.
It is necessary to save file *.php in UTF-8 and without BOM!.
Lena
BCBJ Master
BCBJ Master
 
Posts: 480
Joined: Sun Feb 06, 2011 1:28 pm

Re: POST and UTF-8

Postby rlebeau » Thu Jun 01, 2017 5:21 pm

Lena wrote:I saved all *.php file in UTF-8.


The encoding of the PHP itself is irrelevant. What is important is the encoding of the data it sends back to you.

Lena wrote:What is the correct way to see Russian letters?


Your code is reading the response into a TStringStream that is constructed without a TEncoding specified, so it will use TEncoding::Default, which is ANSI on Windows and UTF-8 on other platforms. If the response uses an encoding that does not match the stream's TEncoding object, the DataString property getter will run into conversion issues with non-ASCII characters. For that reason, I *NEVER* recommend reading an HTTP response into a TStringStream, because then you lose all control of charset handling.

You need to pay attention to the charset that is reported in the response and then decode the response data accordingly.

Had you used Indy''s TIdHTTP instead of Embarcadero's THTTPClient, it would have done the charset handling for you, eg:

Code: Select all
std::unique_ptr<TIdHTTP> aHTTP(new TIdHTTP());
...
String rezult = aHTTP->Post(postlink, paramlist.get());


With THTTPClient, try something more like this:

Code: Select all
std::unique_ptr<THTTPClient> aHTTP(THTTPClient::Create());
...
String rezult = aHTTP->Post(postlink, paramlist.get())->ContentAsString();


Or:

Code: Select all
std::unique_ptr<THTTPClient> aHTTP(THTTPClient::Create());
...
_di_IHTTPResponse Response = aHTTP->Post(postlink, paramlist.get());
std::unique_ptr<TEncoding> Encoding(TEncoding::GetEncoding(Response->ContentCharSet));
String rezult = Response->ContentAsString(Encoding.get());


Lena wrote:It is necessary to save file *.php in UTF-8 and without BOM!.


Or, simply ignore the BOM if it is present:

Code: Select all
...
String rezult = ...
if (!result.IsEmpty())
{
    if (rezult[1] == 0xFEFF)
        result.Delete(1, 1);
    ...
}


Or, just let ParseJSONValue() handle the BOM for you:

Code: Select all
std::unique_ptr<TBytesStream> Response(new TBytesStream());
...
aHTTP->Post(postlink, paramlist.get(), Response.get());
if (Response->Size > 0)
{
    std::unique_ptr<TJSONValue> LJSONValue(TJSONObject::ParseJSONValue(Response->Bytes, 0, true));
    ...
}
Remy Lebeau (TeamB)
Lebeau Software
User avatar
rlebeau
BCBJ Author
BCBJ Author
 
Posts: 1414
Joined: Wed Jun 01, 2005 3:21 am
Location: California, USA

Re: POST and UTF-8

Postby Lena » Fri Jun 02, 2017 8:17 am

Tnak You very much!
Lena
BCBJ Master
BCBJ Master
 
Posts: 480
Joined: Sun Feb 06, 2011 1:28 pm


Return to Technical

Who is online

Users browsing this forum: Baidu [Spider] and 9 guests