简单一句话说明就是: 浏览器跟niconico的交互无非get、post, 中间有通过ssh登录后cookie的添加, 浏览器执行nico页面上的javascript导致cookie的增加或修改, 查询(根据番号)和请求(GET)视频文件时需要带上特定的cookie才能请求成功。
那么哪些操作更改了cookie, 自定义的http请求又要怎么写才能把视频文件搞下来呢? 这些信息通过firebug清楚明了。
从结论往回推倒, 先看下载某个文件的GET请求里带上的cookie是这样:
Cookie nicosid=1376560121.1985494370; WT_FPC=id=183.1.222.67-2718089024.30316956:lv=1376563737475:ss=1376563716164; user_session=user_session_5876440_15212688[人工马赛克]029617; __utma=8292653.1210056190.1376560156.1376619137.1376619257.4; __utmz=8292653.1376560156.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none); nicohistory=sm20892030%3A1376619147%3A1376619352%3Abfccbcfceb510b97%3A5%2Cnm14545546%3A1376618022%3A1376619352%3A71811917e7fba6e4%3A5%2Csm21596305%3A1376560377%3A1376560377%3A8b8813d10b48d5a0%3A1; optimizelySegments=%7B%22281621258%22%3A%22ff%22%2C%22280358707%22%3A%22false%22%2C%22281626694%22%3A%22direct%22%7D; optimizelyEndUserId=oeu1376560363326r0.5227189826598727; optimizelyBuckets=%7B%7D; __utmb=8292653; __utmc=8292653 Host smile-cln15.nicovideo.jp实际测试其实只需要nicosid和nicohistory这两个就足够了。 nicosid怎么获取? 访问niconico首页就会有nicosid写到cookie里。 nicohistory暂时不管。 另外一个假设的前提是我们已经知道视频文件的地址。 下面是代码:
#include#include #include size_t writefunc(void *ptr, size_t size, size_t nmemb, std::string *s){ s->append((char *)ptr, size*nmemb); return size*nmemb;}void TestDownload(){ CURL *curl; CURLcode res; std::string str; curl = curl_easy_init(); if(!curl) { return ; } // 视频文件的地址 curl_easy_setopt(curl, CURLOPT_URL, "http://smile-cln15.nicovideo.jp/smile?m=20892030.52844"); // swf地址, 基本固定, http://res.nimg.jp/swf/player/nicoplayer.swf?ts=xxx 里的ts=xxx也是可以省略的 curl_easy_setopt(curl, CURLOPT_REFERER, "http://res.nimg.jp/swf/player/nicoplayer.swf"); // nicohistory后面的字段也是可以精简的 curl_easy_setopt(curl, CURLOPT_COOKIE, //"nicosid=1376560121.1985494370; nicohistory=sm20892030:1376619147:1376619268:b88d468b9ae66c4a:4,nm14545546:1376618022:1376619268:b3cef30c0550ab80:4,sm21596305:1376560377:1376560377:8b8813d10b48d5a0:1"); "nicosid=1376620271.695811941; nicohistory=sm20892030:1376619147:1376619268:b88d468b9ae66c4a:1"); curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writefunc); curl_easy_setopt(curl, CURLOPT_WRITEDATA, &str); res = curl_easy_perform(curl); AtlTrace("res:%d\n", res); curl_easy_cleanup(curl);}int _tmain(int argc, _TCHAR* argv[]){ TestDownload(); return 0;}
接下来说说怎么获取nicosid, 其实就是http请求nico首页, 然后设置curlib把cookie写到文件里, 最后打开文件看就是了。
void TestGetNicoidCookie(){ CURL *curl; CURLcode res; std::string str; curl = curl_easy_init(); if(!curl) { return ; } curl_easy_setopt(curl, CURLOPT_URL, "http://www.nicovideo.jp/"); curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "c:\\nicoid.txt"); curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "c:\\nicoid.txt"); curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writefunc); curl_easy_setopt(curl, CURLOPT_WRITEDATA, &str); res = curl_easy_perform(curl); AtlTrace("res:%d\n", res); curl_easy_cleanup(curl);}成功后打开c盘的nicoid.ext, 内容:
# Netscape HTTP Cookie File# http://curl.haxx.se/rfc/cookie_spec.html# This file was generated by libcurl! Edit at your own risk..nicovideo.jp TRUE / FALSE 1691980271 nicosid 1376620271.695811941
至于nicohistory, 原来的
nicohistory=sm20892030:1376619147:1376619268:b88d468b9ae66c4a:4,nm14545546:1376618022:1376619268:b3cef30c0550ab80:4,sm21596305:1376560377:1376560377:8b8813d10b48d5a0:1 可裁剪为 nicohistory=sm20892030:1376619147:1376619268:b88d468b9ae66c4a:1sm20892030一看就知道是番号了, 后面三串数字, 前两串对比utma来看应该是时戳, 最后一个估计是前两串时戳的加密验证(前两串修改了一下后对面就返回403了), nicohistory怎么获取我实在没精力查找是那个页面的哪段javascript添加/修改/计算的, 不然既然浏览器知道怎么算出来你肯定也能知道怎么算出来。
再往回查看一下发现前提条件——视频文件的地址还不知道怎么获取, 通过firebug就知道是带上指定番号的POST请求http://flapi.nicovideo.jp/api/getflv页面来获取的, 而这个请求能成功的关键是cookie里带上user_session,假设我们已经得到了user_session:
void TestGetFileAddressFromNicoSMNumber(){ CURL *curl; CURLcode res; std::string str; curl = curl_easy_init(); if(!curl) { return ; } curl_easy_setopt(curl, CURLOPT_URL, "http://flapi.nicovideo.jp/api/getflv"); //基本固定 curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "v=sm20892030"); //番号 curl_easy_setopt(curl, CURLOPT_REFERER, "http://res.nimg.jp/swf/player/nicoplayer.swf"); curl_easy_setopt(curl, CURLOPT_COOKIE, "user_session=user_session_5[码]440_20[人工打码]0019268;" ); curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writefunc); curl_easy_setopt(curl, CURLOPT_WRITEDATA, &str); res = curl_easy_perform(curl); AtlTrace("res:%d\n", res); std::for_each(str.begin(), str.end(), [](const char& a) { AtlTrace("%c", a); }); curl_easy_cleanup(curl);}
成功后控制台输出:
res:0
thread_id=1368834513&l=253&url=http%3A%2F%2Fsmile-cln15.nicovideo.jp%2Fsmile%3Fm%3D20892030.52844&link=http%3A%2F%2Fwww.smilevideo.jp%2Fview%2F20892030%2F5876440&ms=http%3A%2F%2Fmsg.nicovideo.jp%2F16%2Fapi%2F&ms_sub=http%3A%2F%2Fsub.msg.nicovideo.jp%2F16%2Fapi%2F&user_id=[打码]&is_premium=0&nickname=[打码]&time=1376621655816&done=true&ng_rv=1&hms=hiroba01.nicovideo.jp&hmsp=2529&hmst=1000000005&hmstk=1376621715.hFN9Cav8gE8GemURtHo0kuODpB0我们需要的只是url参数而已, 就是上面加粗那段: url=http%3A%2F%2Fsmile-cln15.nicovideo.jp%2Fsmile%3Fm%3D20892030.52844, 明显用了urlencode, 变身一下:
%3A=':' %2F='/' %3F='?' %3D='=' ---------------> http://smile-cln15.nicovideo.jp/smile?m=20892030.52844 这就是我们要下载的地址了。现在的问题是, user_session这个cookie怎么得到? 这个cookie要你通过账号密码成功登录nico后才能拿到。 而nico的登录用的是https, 即SSH加密的http请求, 代码:
void TestLoginToGetUserSession(){ CURL *curl; CURLcode res; std::string str; curl = curl_easy_init(); if(!curl) { return ; } curl_easy_setopt(curl, CURLOPT_URL, "https://secure.nicovideo.jp/secure/login?site=niconico"); curl_easy_setopt(curl, CURLOPT_USE_SSL, (long)CURLUSESSL_ALL); curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "c:\\user_session.txt"); curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "c:\\user_session.txt"); curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, FALSE); curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, FALSE); curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "next_url=&mail_tel=填入你账号(邮箱)&password=填入你密码"); curl_easy_setopt(curl, CURLOPT_REFERER, "https://secure.nicovideo.jp/secure/login_form"); curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writefunc); curl_easy_setopt(curl, CURLOPT_WRITEDATA, &str); res = curl_easy_perform(curl); AtlTrace("res:%d\n", res); curl_easy_cleanup(curl);}成功后打开c盘的user_session.txt, 内容:
# Netscape HTTP Cookie File# http://curl.haxx.se/rfc/cookie_spec.html# This file was generated by libcurl! Edit at your own risk..nicovideo.jp TRUE / FALSE 1691982237 nicosid 1376622237.1622763304secure.nicovideo.jp FALSE /secure/ FALSE 1 user_session deletedsecure.nicovideo.jp FALSE / FALSE 1 user_session deleted.nicovideo.jp TRUE / FALSE 1379214237 user_session user_session_5876440_13[人工打码]165630.nicovideo.jp TRUE / FALSE 1 repair_history deletedok, user_session也拿到了。 那还有什么问题? 对, nicohistory, 这个就交给你们了(拇指)。