| too much http traffic around the external torrents indexed ! [message #1814] |
Tue, 19 February 2008 21:11  |
DrAgon  Messages: 192 Registered: August 2007 |
Super-Seeder Bug hunter |
|
|
There are lots of traffic on torrents that are not local to the tracker..
way too much...this ain't necessary..I have to find out what the hell is going on.... why are there traffic on torrents that are not hosted locally from various IPs ..
they are there....even when I have autoscrape turned off..so that way I'm sure..there are not outgoing http fopen sockets..
http logs are fine with hits/scrape/announce on local torrents..but these are traffic..(incoming) coming in on http but not showing in apache logs and are socket based perhaps...and for getting info on external torrents ..
I donno..but I'll find out..
|
|
|
|
| Re: too much http traffic around the external torrents indexed ! [message #1839 is a reply to message #1815] |
Mon, 10 March 2008 02:05   |
DrAgon  Messages: 192 Registered: August 2007 |
Super-Seeder Bug hunter |
|
|
okay..here is what I see..
Why should I get so much traffic on external torrents listed on tracker index? it don't even have dht on those torrents..
Why should my tracker care about how much he is downloaded or upload or incompletel on a specific external torrent?
This is tons of unnecessary traffic. It is not RIGHT! something is wrong..
Here is just one example of the dump of network sniffing, I replaced my tracker info with "tracker" and client ip with "client" and name of torrent to "torrent description which don't even belong to my local tracker!!!!
tracker -> client HTTP 20058:completei0e10:downloadedi1e10:incompletei0ee20:&.C Q
tracker -> client HTTP (body)
client -> tracker HTTP C port=2413
tracker -> client HTTP Torrent description(DivX) avi8:completei3e10:downloadedi145e10:incompletei0ee20:
client -> tracker HTTP C port=2413
tracker -> client HTTP (body)
tracker -> client HTTP torrent description (Ch 4) 2008 02 15 xvid8:completei18e10:downloadedi29e10:in
tracker -> client HTTP tei19e10:downloadedi34e10:incompletei27ee20:]C`&Y
tracker -> client HTTP de Barl8:completei16e10:downloadedi30e10:incompletei0ee20:1
client -> tracker HTTP C port=2413
tracker -> client HTTP (body)
it seems like I get all the peers traffic on all torrents even if external...something ain't right...help if you can..
I'm talking about tons of packets that I should not see or the tracker should even care about!!
Are these sources scraping external torrents stats off of my tracker!!? why?
I even see what other torrents they have on thier bittorent client ..that I don't have anything to do with them!? not even on the website torrent index list!?
this is some wierd stuff..
[Updated on: Mon, 10 March 2008 02:57] Report message to a moderator
|
|
|
| Re: too much http traffic around the external torrents indexed ! [message #1840 is a reply to message #1839] |
Mon, 10 March 2008 03:20   |
|
is your announce being added to the torrents?
sounds like it a torrent client issue besides your tracker should be kicking these announce's and scrapes for torrents that arnt yours back with error saying not regesterd with this tracker.
It could also be some people adding your announce to there torrent client so that it well announce to your site. I know that I have done this a few times my self . but that was to seed a torrent on that site that needed seeding. But if you just add the announce to the torrent then it changes the info_hash and renders the torrent useless to the first site and would have to be reuploaded to both sites. have you checked the tracker list of some of these torrents to see if your site was added to the torrent?
[Updated on: Mon, 10 March 2008 03:29] Report message to a moderator
|
|
|
|
|
|
| Re: too much http traffic around the external torrents indexed ! [message #1844 is a reply to message #1843] |
Tue, 11 March 2008 07:05   |
DrAgon  Messages: 192 Registered: August 2007 |
Super-Seeder Bug hunter |
|
|
where is clean.php?
I see ob_clean in announce.php and cleanup.php in include
well..it seems like my tracker is scraping off the bittorrent clients.. instead of other trackers!
I look into the code to see what's going on!....unless u find something interesting....sooner..let me know
update:
I have a feeling announce.php response is wrong...
either infohash decoding is incorrect or corrupted tables..somewhere..or maybe info written/read to the tables during announce/scraping is wrong..
I might have to write a script/routine to grab infohash decoded request from client and and see why announce.php response are about the external torrents stats!
[Updated on: Tue, 11 March 2008 09:18] Report message to a moderator
|
|
|
|
|
|
|
|
| Re: too much http traffic around the external torrents indexed ! [message #1999 is a reply to message #1928] |
Sat, 10 May 2008 06:13   |
DrAgon  Messages: 192 Registered: August 2007 |
Super-Seeder Bug hunter |
|
|
I also tried to optimize scrape.php to only respond to local torrents and not external torrents..
I think this has been the major source of traffic..that is not needed..lodp also will benefit from this ....he asked me about it today..I have him..refer to this post..
with the changes..I ran few tests.. and seeding/leeching local torents looked okay..and seem to be working..
but not quite sure about it yet...
azures scrape status says ok..and utorrent finally scrape status said ok.. when I stop torrent! I'll run more tests later..
anyways...please check the code and let me know if it is correct..red fonts are additions.
I'm not sure about the two lines with chk in it , either.
if (count($info_hash_arr) < 1) $sql = "SELECT info_hash, tracker, seeders AS complete, com
pleted AS downloaded, leechers AS incomplete, name FROM ".$db_prefix."_torrents ORDER BY i
d;";
else $sql = "SELECT info_hash, tracker, seeders AS complete, completed AS downloaded, leec
hers AS incomplete, name FROM ".$db_prefix."_torrents WHERE ".hash_where("info_hash", $inf
o_hash_arr);^M
^M
if (! $res = $db->sql_query($sql)) err("SQL ERROR: ".$sql);^M
$chk = $db->sql_fetchrow($sql);
if ($chk["tracker"] != "")err("Torrent not registered with this tracker.");if ($db->sql_numrows($res) < 1) err("Torrent not registered with this tracker.");^M
^M
$r .= "d5:filesd";
while ($row = $db->sql_fetchrow($res)) {^M
if ($row["tracker"] == ""){
$r .= "20:" . hash_pad($row["info_hash"]) . "d" .^M
"4:name". strlen($row["name"]) .":". $row["name"].^M
"8:completei". $row["complete"] ."e".^M
"10:downloadedi". $row["downloaded"] ."e".^M
"10:incompletei". $row["incomplete"] ."e".^M
# "tracker". $row["tracker"] ."e".
"e";^M
}
}^M
[Updated on: Sat, 10 May 2008 06:22] Report message to a moderator
|
|
|
|
| Re: too much http traffic around the external torrents indexed ! [message #2001 is a reply to message #1999] |
Sat, 10 May 2008 14:08   |
|
| DrAgon wrote on Sat, 10 May 2008 01:13 | I also tried to optimize scrape.php to only respond to local torrents and not external torrents..
I think this has been the major source of traffic..that is not needed..lodp also will benefit from this ....he asked me about it today..I have him..refer to this post..
with the changes..I ran few tests.. and seeding/leeching local torents looked okay..and seem to be working..
but not quite sure about it yet...
azures scrape status says ok..and utorrent finally scrape status said ok.. when I stop torrent! I'll run more tests later..
anyways...please check the code and let me know if it is correct..red fonts are additions.
I'm not sure about the two lines with chk in it , either.
if (count($info_hash_arr) < 1) $sql = "SELECT info_hash, tracker, seeders AS complete, com
pleted AS downloaded, leechers AS incomplete, name FROM ".$db_prefix."_torrents ORDER BY i
d;";
else $sql = "SELECT info_hash, tracker, seeders AS complete, completed AS downloaded, leec
hers AS incomplete, name FROM ".$db_prefix."_torrents WHERE ".hash_where("info_hash", $inf
o_hash_arr);^M
^M
if (! $res = $db->sql_query($sql)) err("SQL ERROR: ".$sql);^M
$chk = $db->sql_fetchrow($sql);
if ($chk["tracker"] != "")err("Torrent not registered with this tracker.");if ($db->sql_numrows($res) < 1) err("Torrent not registered with this tracker.");^M
^M
$r .= "d5:filesd";
while ($row = $db->sql_fetchrow($res)) {^M
if ($row["tracker"] == ""){
$r .= "20:" . hash_pad($row["info_hash"]) . "d" .^M
"4:name". strlen($row["name"]) .":". $row["name"].^M
"8:completei". $row["complete"] ."e".^M
"10:downloadedi". $row["downloaded"] ."e".^M
"10:incompletei". $row["incomplete"] ."e".^M
# "tracker". $row["tracker"] ."e".
"e";^M
}
}^M
|
this is all wrong and you are about to kill your scrape
|
|
|
| Re: too much http traffic around the external torrents indexed ! [message #2002 is a reply to message #1814] |
Sat, 10 May 2008 15:43   |
lodp  Messages: 336 Registered: June 2006 |
Releaser |
|
|
I checked my webalizer stats and discovered that 90% of my outgoing data on the server originates from scrape.php. That's a whole frickin lot.
My issue is not so much people scraping for external torrents, but the length of the response they get. I checked my apache access logs, and they say the response is about 340kb.
Are you guys getting the same numbers in your logs? Isn't 340KB way too long? Isn't the response only supposed to consist of a short string (Seeds, leechers, downloaded, infohash).
I tried to enter debug code into scrape.php, to have it mail the response to me (like announce.php can), but not success so far..
EDIT OK, i think i can see what the problem is now. It's serving a full scrape for ALL torrents, even when the client only scrapes for one infohash.
The code is supposed to be able to distinguish between scrapes with multiple hashes and single ones, but for some reason it always assumes multiple:
if (count($info_hash_arr) < 1) $sql = "SELECT info_hash, seeders AS complete, completed AS downloaded, leechers AS incomplete, name FROM ".$db_prefix."_torrents ORDER BY id;";
else $sql = "SELECT info_hash, seeders AS complete, completed AS downloaded, leechers AS incomplete, name FROM ".$db_prefix."_torrents WHERE ".hash_where("info_hash", $info_hash_arr);
[Updated on: Sat, 10 May 2008 16:08] Report message to a moderator
|
|
|
| Re: too much http traffic around the external torrents indexed ! [message #2003 is a reply to message #2002] |
Sat, 10 May 2008 17:07   |
|
please replace your scrape.php with
<?php
/*
*-------------------------------phpMyBitTorrent--------------------------------*
*--- The Ultimate BitTorrent Tracker and BMS (Bittorrent Management System) ---*
*-------------- Created By Antonio Anzivino (aka DJ Echelon) --------------*
*------------- http://www.p2pmania.it -------------*
*------------ Based on the Bit Torrent Protocol made by Bram Cohen ------------*
*------------- http://www.bittorrent.com -------------*
*------------------------------------------------------------------------------*
*------------------------------------------------------------------------------*
*-- This program is free software; you can redistribute it and/or modify --*
*-- it under the terms of the GNU General Public License as published by --*
*-- the Free Software Foundation; either version 2 of the License, or --*
*-- (at your option) any later version. --*
*-- --*
*-- This program is distributed in the hope that it will be useful, --*
*-- but WITHOUT ANY WARRANTY; without even the implied warranty of --*
*-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the --*
*-- GNU General Public License for more details. --*
*-- --*
*-- You should have received a copy of the GNU General Public License --*
*-- along with this program; if not, write to the Free Software --*
*-- Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA --*
*-- --*
*------------------------------------------------------------------------------*
*------ ©2005 phpMyBitTorrent Development Team ------*
*----------- http://phpmybittorrent.com -----------*
*------------------------------------------------------------------------------*
*/
require_once("include/config_lite.php");
if (eregi("(Mozilla|Opera|Lynx|Netscape)",$HTTP_SERVER_VARS["HTTP_USER_AGENT"])) {
die("<html><head><title>Error!</title></head><body><h3>Sorry, but this file is not suitable for browsers.</h3></body></html>");
}
if ($stealthmode) die();
function hash_where($name, $hash_arr) {
$new_hash_arr = Array();
foreach ($hash_arr as $hash) {
array_push($new_hash_arr,"'".urldecode ( $hash)."'");
}
return $name." IN ( ".implode(",",$new_hash_arr)." )";
}
function err($msg) {
benc_resp_raw("d14:failure reason". strlen($msg) .":". $msg);
die();
}
function hash_pad($hash) {
return str_pad($hash, 20);
}
function benc_resp_raw($x) {
header("Content-Type: text/plain");
header("Pragma: no-cache");
echo $x;
}
//$info_hash_array = array('info_hash');
preg_match_all('/info_hash=([^&]*)/i', $_SERVER["QUERY_STRING"], $info_hash_array);
$info_hash_array = str_replace('info_hash=','',$info_hash_array[0]);
//print_r($info_hash_array);
if (count($info_hash_array) < 1) $sql = "SELECT tracker, info_hash, seeders AS complete, completed AS downloaded, leechers AS incomplete, name FROM ".$db_prefix."_torrents WHERE tracker IS NULL AND exeem IS NOT NULL ORDER BY id;";
else $sql = "SELECT info_hash, seeders AS complete, completed AS downloaded, leechers AS incomplete, name FROM ".$db_prefix."_torrents WHERE ".hash_where("info_hash", $info_hash_array);
if (! $res = $db->sql_query($sql)) err("SQL ERROR: ".$sql);
if ($db->sql_numrows($res) < 1) err("Torrent not registered with this tracker." . $sql ."");
$r .= "d5:filesd";
while ($row = $db->sql_fetchrow($res)) {
$r .= "20:" . hash_pad($row["info_hash"]) . "d" .
"4:name". strlen($row["name"]) .":". $row["name"].
"8:completei". $row["complete"] ."e".
"10:downloadedi". $row["downloaded"] ."e".
"10:incompletei". $row["incomplete"] ."e".
"e";
}
$r .= "e";
if ($announce_interval > 0 OR $announce_interval_min > 0) {
$r .= "5:flagsd20:min_request_intervali".max($announce_interval,$announce_interval_min)."ee";
}
$r .= "e";
ob_clean();
benc_resp_raw($r);
$db->sql_query("",END_TRANSACTION);
if (!$db->persistency) $db->sql_close();
die();
?>
[Updated on: Sat, 10 May 2008 17:07] Report message to a moderator
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Re: too much http traffic around the external torrents indexed ! [message #2042 is a reply to message #1814] |
Sat, 17 May 2008 20:21  |
Revan  Messages: 125 Registered: April 2007 |
Super-Seeder |
|
|
Just realized that the same bug that occurs in announce for some clients - missing key, will probably occur here too...
So how about adding something like this before the line with preg_match_all?
if (isset($_GET["passkey"])) $passkey = $_GET["passkey"];
else $passkey = "";
if (strpos($passkey, "?"))
{
$tmp = substr($passkey , strpos($passkey , "?"));
$passkey = substr($pid , 0,strpos($passkey , "?"));
$tmpname = substr($tmp, 1, strpos($tmp, "=")-1);
$tmpvalue = substr($tmp, strpos($tmp, "=")+1);
$_GET[$tmpname] = $tmpvalue;
}
if ($passkey!="") $_SERVER["QUERY_STRING"]=substr($_SERVER["QUERY_STRING"],strlen("?passkey=$passkey"));
else $_SERVER["QUERY_STRING"]=$_SERVER["QUERY_STRING"];
Let me know what you think...
|
|
|