Delete duplicate entry from text file

[jack001]

hi all

I need to know how can i delete duplicate lines in a text file which is on the my server with help of php.


please help me

thanks in advance
Jack

[MrCoder]

Import the file in to an array.
Create a new array from using array_unique() on the existing one.
Save the new array to a file.

[jack001]

Dear MrCoder

I am new to php, if u can help me with an exmaple, it will be great.

thanks in advance
jack


Import the file in to an array.
Create a new array from using array_unique() on the existing one.
Save the new array to a file.

[MrCoder]

$lines = file("myfile.txt"); // Get the file as an array
$lines = array_unique($lines); // Merge all duplicate lines

// Save as a new file
$file = fopen("mynewfile.txt", "w");
fwrite($file, implode("\r\n", $lines));
fclose($file);

[jack001]

I created mynewfile.txt on my server but it gives following error:

fwrite(): supplied argument is not a valid stream resource


$lines = file("myfile.txt"); // Get the file as an array
$lines = array_unique($lines); // Merge all duplicate lines

// Save as a new file
$file = fopen("mynewfile.txt", "w");
fwrite($file, implode("\r\n", $lines));
fclose($file);

[MrCoder]

The above code creates the file "mynewfile.txt", just make sure the folder that you execute the script from is CHMOD 777

[jack001]

I deleted the file which i created manually and set CHMOD 777 for the folder where this php tile is located, but still same error.

it's creating new_signup.txt but no data in it and same error of fwrite

[MrCoder]

Check this out (http://uk2.php.net/manual/en/function.fopen.php)

Was the file created by fopen()?

How do you have your server set-up?
Without knowing more I'm not sure I can help.


Edit: Try running the example code here (http://uk2.php.net/manual/en/function.fwrite.php).

[jack001]

the below code just writing "Array" text in the new file

lines = file("myfile.txt"); // Get the file as an array
$lines = array_unique($lines); // Merge all duplicate lines

// Save as a new file
$file = fopen("mynewfile.txt", "w");
fwrite($file, implode("\r\n", $lines));
fclose($file);

[MrCoder]

<?php
$lines = file("myfile.txt"); // Get the file as an array
$lines = array_unique($lines); // Merge all duplicate lines

// Save as a new file
$file = fopen("mynewfile.txt", "w");
fwrite($file, implode("", $lines));
fclose($file);
?>


Just got home from work and tested it on my web server, the above worked perfectly for me, notice I removed the "\r\n" from the implode since file() maintained the line breaks.

I do not understand how this cannot be working for you, it is only a small bit of code :)

Use the following bit of PHP to dump some debug data insted.


<?php
$lines = file("myfile.txt"); // Get the file as an array
echo "<pre>-- Lines --\r\n";
var_dump($lines);

$lines = array_unique($lines); // Merge all duplicate lines
echo "\r\n-- Lines after array_unique() --\r\n";
var_dump($lines);

// Save as a new file
$file = fopen("mynewfile.txt", "w");

echo "\r\n-- File --\r\n";
var_dump($file);

echo "\r\n-- Lines Implode --\r\n";
var_dump(implode("", $lines));

echo "</pre>";

fwrite($file, implode("", $lines));
fclose($file);
?>


If you still have problem then copy paste the results from the var_dump()'s in your next reply.

myfile.txt contents

this is line 1
this is line 2
this is line 3
this is line 1
this is line 4


mynewfile.txt contents

this is line 1
this is line 2
this is line 3
this is line 4


I get the following from the var_dump()'s using my "myfile.txt"

-- Lines --
array(5) {
[0]=>
string(15) "this is line 1
"
[1]=>
string(15) "this is line 2
"
[2]=>
string(15) "this is line 3
"
[3]=>
string(15) "this is line 1
"
[4]=>
string(14) "this is line 4"
}

-- Lines after array_unique() --
array(4) {
[0]=>
string(15) "this is line 1
"
[1]=>
string(15) "this is line 2
"
[2]=>
string(15) "this is line 3
"
[4]=>
string(14) "this is line 4"
}

-- File --
resource(4) of type (stream)

-- Lines Implode --
string(59) "this is line 1
this is line 2
this is line 3
this is line 4"

[jack001]

OK, The problem is solved now. I am getting what I want....

Thanks a ton Mr. MrCoder

[MrCoder]

Glad I could help.