How SNEAK deals w/ unicode . . .

[Replies] [Reply] [View by Thread] [Help]
[Back To SNEAK Development Forum]

Cpt SAJChurchey

Well,

From what I can gather, the PHP version of sneak will convert UTF-8 into binary if it is put in as an "ASCII" value. As far as UTF-16 is concerned, it substitutes the character in the text field w/ what I guess is an HTML code:

ÀԳՉ is changed to ÀԳՉ

Չ => Չ
Գ => Գ

Then, SNEAK analyzes each of the seven characters in the substituted code and adds them to the binary string that is reported.

ÀԳՉ => 11000000 00100110 00100011 00110001 00110011 00110011 00110001 00111011 00100110 00100011 00110001 00110011 00110101 00110011 00111011

This is incorrect b/c UTF-16 is not made up of 7 bytes but 2, but then again this is an ASCII2Bin function not a UTF162Bin function. So we need to handle this special case in the same manner that SNEAK does, or should we improve the conversion capabilities if we can?

Cpt SAJChurchey
C/O of Editorial
OSI Staff edit0r
OSI Feedback Representative

Replies:

Desktop Version - CinC snarkles 2005-02-15 17:19:31
- Java - HTML Entities Encr/decr - draft 1 - Alpha Cpt FantaSy 2006-01-03 05:04:05
  - Oh, you've done the HTML encode/decode - Beta Cpt SAJChurchey 2006-02-05 05:33:58
- Java - RSA Implementation - Lt CyberTNT 2005-03-21 03:08:22
  - [DONE] - Lt CyberTNT 2005-03-24 14:10:17
- Java - MD5 - First Draft - Maj anvar 2005-03-14 19:01:39
  - The working version (TM) :) - Maj anvar 2005-03-18 14:33:00
    - RE: The working version (TM) :) - Lt CyberTNT 2005-03-18 15:05:54
      - RE: The working version (TM) :) - Maj anvar 2005-03-18 16:02:48
        You cant use String.format... thats 1.5 only -nt- - Ret. Gen D-Cypell 2005-03-18 17:04:23
        2 questions - Maj anvar 2005-03-18 17:19:21
        RE: 2 questions - Tr flamebalrog 2005-03-24 22:41:05
        RE: 2 questions - CinC snarkles 2005-03-18 17:41:40
        RE: 2 questions - Lt CyberTNT 2005-03-18 17:41:38
  - short note about it - Maj anvar 2005-03-15 18:27:03
    - RE: short note about it - Lt CyberTNT 2005-03-17 01:02:31
    - RE: short note about it - Tr flamebalrog 2005-03-16 23:20:17
  - Some things . . . - Cpt SAJChurchey 2005-03-14 19:22:49
- Which GUI Library? - Cpt SAJChurchey 2005-03-08 04:48:45
  - shouldnt be a problem nt - Lt CyberTNT 2005-03-08 21:43:58
- InvalidArgumentExceptions--How should we handle it - Cpt SAJChurchey 2005-02-28 19:31:39
  - Yeah, that would work... - CinC snarkles 2005-03-01 21:51:01
- Java - Bin2Text - Second Draft - Cpt SAJChurchey 2005-02-24 20:22:31
  - Added to SVN - Cpt SAJChurchey 2005-03-14 19:28:27
  - RE: Java - Bin2Text - Second Draft - Tr flamebalrog 2005-02-26 02:06:30
    - Modified (Exceptions Included) - Cpt SAJChurchey 2005-02-27 20:11:58
      - I'm evil... - Tr flamebalrog 2005-03-06 00:27:37
        You learn something new everyday (revision) - Cpt SAJChurchey 2005-03-07 21:37:56
    - Bit shifts should be faster, thanx -nt- - Cpt SAJChurchey 2005-02-26 19:19:48
      - np -nt- - Tr flamebalrog 2005-02-27 22:55:00
- Java - Backwards - Second Draft - Cpt SAJChurchey 2005-02-24 20:18:01
  - Added to SVN - Cpt SAJChurchey 2005-03-14 19:29:48
  - RE: Java - Backwards - Second Draft - Ret. Gen D-Cypell 2005-02-26 19:22:23
    - and factor trim() as well ? -nt- - Tr flamebalrog 2005-02-27 01:26:55
    - In this case . . . - Cpt SAJChurchey 2005-02-26 23:18:34
  - RE: Java - Backwards - Second Draft - Lt dopel 2005-02-26 16:51:44
    - I had to take str=str.trim() out - Cpt SAJChurchey 2005-02-26 19:16:27
    - Nope.... - Ret. Gen D-Cypell 2005-02-26 19:14:49
  - RE: Java - Backwards - Second Draft - Tr flamebalrog 2005-02-26 02:15:52
    - Resolved - Cpt SAJChurchey 2005-02-27 19:54:13
    - RE: Java - Backwards - Second Draft - Ret. Gen D-Cypell 2005-02-26 19:13:56
      - Thx for your input [bit more] - Tr flamebalrog 2005-02-27 23:01:18
    - RE: Java - Backwards - Second Draft - Lt dopel 2005-02-26 18:38:08
- Java - Hex2Bin - First Draft - Cpt SAJChurchey 2005-02-24 20:14:07
- Java - Hex2Text - First Draft - Cpt SAJChurchey 2005-02-24 20:07:17
  - What about replacing hex2decimal() by... - Tr flamebalrog 2005-03-06 00:15:30
    - correction made - Cpt SAJChurchey 2005-03-07 21:41:14
- Java - Text2Hex - First Draft - Cpt SAJChurchey 2005-02-24 20:04:54
- Java - ASCII2Bin - Second Draft - Cpt SAJChurchey 2005-02-21 21:35:04
  - Added to SVN - Cpt SAJChurchey 2005-03-14 19:31:00
  - One thing . . . - Cpt SAJChurchey 2005-02-22 03:17:07
    - --regex - Cpt SAJChurchey 2005-02-22 22:28:15
      - NPE Fix - Cpt SAJChurchey 2005-02-23 19:42:04
- For people writing drafts.... - Ret. Gen D-Cypell 2005-02-18 14:41:45
  - I can do these... - CinC snarkles 2005-02-19 15:15:23
    - OK. Here's a start anyway... - CinC snarkles 2005-02-20 01:00:36
      - Eeek. Update... - CinC snarkles 2005-02-21 21:21:18
        Binary -> ASCII Test Cases - Cpt SAJChurchey 2005-02-22 01:52:49
- Java - ASCII2Bin - First Draft - Ker Socrat 2005-02-17 20:09:14
  - RE: Java - ASCII2Bin - First Draft - Ret. Gen D-Cypell 2005-02-18 14:32:32
    - >How SNEAK deals w/ unicode . . . - Cpt SAJChurchey 2005-02-19 16:57:24 <
      - By all means, go for improvement, imho... - CinC snarkles 2005-02-19 22:29:17
        UTF162Bin instead of ASCII2Bin - Cpt SAJChurchey 2005-02-20 04:24:09
    - Taking UTF-8 and UTF-16 into consideration . . . - Cpt SAJChurchey 2005-02-19 02:07:32
      - RE: Taking UTF-8 and UTF-16 into consideration . . . - Ker Socrat 2005-02-19 03:16:21
        Where can I find the php version? (revision included) - Cpt SAJChurchey 2005-02-19 07:20:20
        PHP version - CinC snarkles 2005-02-19 14:34:09
        Can we add this to MOTD for quick access? -nt- - Cpt SAJChurchey 2005-02-22 02:08:43
        One more thing . . . - Cpt SAJChurchey 2005-02-19 07:45:13
        After running the test cases . . . - Cpt SAJChurchey 2005-02-21 19:29:29
        Oh. . . and the changes I made this round - Cpt SAJChurchey 2005-02-21 19:31:53
        Small clarifications - CinC snarkles 2005-02-21 20:25:23
        Hopefully, it'll work now . . . - Cpt SAJChurchey 2005-02-21 20:56:32
        
        RE: One more thing . . . - Ker Socrat 2005-02-19 16:47:32
        also, - Ker Socrat 2005-02-19 16:53:36
        vim baby! - Cpt SAJChurchey 2005-02-19 16:59:31
  - RE: Java - ASCII2Bin - First Draft - Lt CyberTNT 2005-02-17 21:06:22
  - RE: Java - ASCII2Bin - First Draft - Cpt SAJChurchey 2005-02-17 20:22:53
    - RE: Java - ASCII2Bin - First Draft - Ker Socrat 2005-02-18 00:23:25
      - It makes sense - Cpt SAJChurchey 2005-02-18 01:46:36
    - can you prove.... - Lt CyberTNT 2005-02-17 21:04:00
      - An explanation . . . - Cpt SAJChurchey 2005-02-18 01:41:23
        maybe DC can clear up things... - Lt CyberTNT 2005-02-18 09:53:05
        I will try... - Ret. Gen D-Cypell 2005-02-18 15:16:40
        absolutly - Lt dopel 2005-02-18 19:48:47
        I do not think it effects the ease of read - Cpt SAJChurchey 2005-02-18 20:27:02
        not to mention . . . - Cpt SAJChurchey 2005-02-18 20:29:47
        
        Version using StringBuffer - Cpt SAJChurchey 2005-02-18 16:54:23
- Java - rot13 method v1.0 - pre-approval - Mar Tacheon 2005-02-15 20:12:52
  - RE: Java - rot13 method v1.0 - pre-approval - Lt CyberTNT 2005-02-16 10:53:39
    - RE: Java - rot13 method v1.0 - pre-approval - Ker Socrat 2005-02-16 11:58:31
      - RE: Java - rot13 method v1.0 - pre-approval - Lt CyberTNT 2005-02-16 13:24:36
        RE: Java - rot13 method v1.0 - pre-approval - Ker Socrat 2005-02-16 13:33:50
        RE: Java - rot13 method v1.0 - pre-approval - Ret. Gen D-Cypell 2005-02-16 15:53:18
        RE: Java - rot13 method v1.0 - pre-approval - Ker Socrat 2005-02-16 18:13:29
        lol oh geez... don't listen to me.. :P - CinC snarkles 2005-02-16 22:49:35
        You dont give yourself credit... - Ret. Gen D-Cypell 2005-02-16 23:33:07
        Wow! - CinC snarkles 2005-02-17 02:52:51
        
        RE: Java - rot13 method v1.0 - pre-approval - Lt CyberTNT 2005-02-16 14:21:44
- Java - l33tt4lk method v1.0 - pre-approval - Ker Socrat 2005-02-15 18:30:25
  - RE: Java - l33tt4lk method v1.0 - pre-approval - Lt CyberTNT 2005-02-15 19:08:14

Guest:
Subject:
Message:	On 2005-02-19 16:57:24, SAJChurchey wrote >Well, > >From what I can gather, the PHP version of sneak will convert UTF-8 into binary if it is put in as an "ASCII" value. As far as UTF-16 is concerned, it substitutes the character in the text field w/ what I guess is an HTML code: > >ÀԳՉ is changed to ÀԳՉ > >Չ => Չ >Գ => Գ > >Then, SNEAK analyzes each of the seven characters in the substituted code and adds them to the binary string that is reported. > >ÀԳՉ => 11000000 00100110 00100011 00110001 00110011 00110011 00110001 00111011 00100110 00100011 00110001 00110011 00110101 00110011 00111011 > >This is incorrect b/c UTF-16 is not made up of 7 bytes but 2, but then again this is an ASCII2Bin function not a UTF162Bin function. So we need to handle this special case in the same manner that SNEAK does, or should we improve the conversion capabilities if we can? >
Signature:
Optional Image Link:
http://

CyberArmy::Forum v0.6
Generated In 0.02294 seconds