Non-English RegExp for removing non-alphanumeric

**Sylvan012** · 02-24-2011, 10:03 AM

I have used this script:

Code:

someString.replace(/[^A-Za-z0-9 .]/g, '')

...many times to remove non-alphanumeric and non "." and " " characters but am having to re-think its use as I start working on non-American English languages for string replacement. The reason for this is that this RegExp also pulls out special characters such as "ó" and "ñ". I'm not certain, but I think it would also remove all double-byte characters such as various Asian-language words.

Has anyone run into this problem and have they found a simple coding solution to catch all non-English special characters?

Yours,
Dave

**Kor** · 02-24-2011, 10:45 AM

Yes, it is possible, but you should specify the special characters you want to allow. And use the suitable utf charcode, probably utf-8 in your case.

Code:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<title>Untitled Document</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta http-equiv="Content-Style-Type" content="text/css">
<meta http-equiv="Content-Script-Type" content="text/javascript">
<script type="text/JavaScript">
function valid(f) {
f.value=f.value.replace(/[^A-z&#199;&#209;Q&#192;&#193;&#200;&#201;&#205;&#204;&#207;&#211;&#210;&#218;&#217;&#220;]/ig,'');
} 
</script>
</head>
<body><br>
<form id="myform" action="">
<input name="mytext" type="text" onkeyup="valid(this)" onblur="valid(this)">
</form>
</body>
</html>

**Sylvan012** · 02-24-2011, 10:49 AM

So what you're suggesting is build a library of uppercase and lowercase special characters and using that for the RegExp?

Makes sense...

Is there a single or simpler command, however, that would include double-byte or all non-English-but-still-alphanumeric characters that you know of?

Yours,
Dave

**Kor** · 02-24-2011, 11:08 AM

Originally Posted by Sylvan012

Is there a single or simpler command, however, that would include double-byte or all non-English-but-still-alphanumeric characters that you know of?

Well, somehow yes, but the RegExp range is not related with the single or double byte characters, it is related with the ASCII range of them. A range like [a-z] covers the ASCII from 97(a) to 122(z). If your special characters you want to allow are in a continuous range, you may use only the first and the last term of the range, as in extended ASCII.

**Sylvan012** · 02-24-2011, 11:10 AM

Originally Posted by Kor

Well, somehow yes, but the RegExp range is not related with the single or double byte characters, it is related with the ASCII range of them. A range like [a-z] covers the ASCII from 97(a) to 122(z). If your special characters you want to allow are in a continuous range, you may use only the first and the last term of the range.

Ooohhh... Ok, that's an excellent observation I'd not thought of.

Off-hand, do you know of a good online resource that could provide an incremental list of all such characters?

I'll search for my own, certainly, but if you've got experience with a good resource like that, I thought I'd ask.

Yours,
Dave

**Kor** · 02-24-2011, 11:33 AM

http://www.cdrummond.qc.ca/cegep/inf...iles/ascii.htm

For instance [Ç-Ñ] should cover all the characters from ASCII extended 128 = Ç to 165 =Ñ.

**Sylvan012** · 02-24-2011, 11:47 AM

Originally Posted by Kor

http://www.cdrummond.qc.ca/cegep/inf...iles/ascii.htm

For instance [Ç-Ñ] should cover all the characters from ASCII extended 128 = Ç to 165 =Ñ.

Whoa... Ok, that's pretty useful and awesome! Thank you!

**Kor** · 02-24-2011, 12:49 PM

If you are interested in the matter, here's some additional information about handling special characters, ASCII codes, foreign alphabets. Unicode and Regular Expressions.

You said something about Asian characters and double-byte characters. In fact there are several way of encoding (encoding and characters set are different animals): 1, 2, 3, or 4 bytes per code point (or combination). Code points are a sort of a mapping between characters and numbers. There are "mysterious" name for those Unicode variants (UTF-36 covers 4 bytes per code point, while UTF-8 covers 1,2,3, and 4 bytes per code point

)

Take care, also about the difference between bit and byte

When in comes about Regular Expressions, a range of characters can be created not only upon their ASCII decimal value, but also on their Unicode or Hexa values. A detailed explanation/standard:

http://unicode.org/reports/tr18/

And an additional note about Regular Expressions in JavaScript. Usually people think that RegExp notation is universal. Well, it is more or less a truth. There are some small differences (in fact few incomplete implementations) from a language to another. In JavaScript, the RegExp are similar with those in Pearl.

Thread: Non-English RegExp for removing non-alphanumeric

Thread Tools

Display

Non-English RegExp for removing non-alphanumeric

Thread Information

Users Browsing this Thread

Tags for this Thread

Bookmarks

Bookmarks

Posting Permissions