Jump to content

Welcome to rile5.com
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. This message will be removed once you have signed in.


Login to Account Create an Account
Photo

[THREADING!]Fastest(?) Item downloader [Python]


  • Please log in to reply
18 replies to this topic

#1
OFFLINE   albertvision

albertvision

    User

  • Members
  • 1014 posts
  • Programming:PHP, MySQL, HTML, CSS, JavaScript, AJAX, jQuery

I wrote this in my spare time, but it's currently the fastest downloader(I think) for CP items, that exists.

 

Why would I use that?

Threading. This project uses threads for downloading, meaning you will download with 3 times(or less) the speed. It will definitely be faster!

 

Why isn't it an exe?

Because fuck exe's. Also, compiler plain out refused to compile my shit.

 

Screenshot?

c1cv.png

How do I use it?

Install python 3.3(or higher). Create a folder(say items), and create these subfolders:

items/sprites

items/paper

items/icons

 

Create a file, call it run.py(or something ending with py)

items/run.py

 

Run run.py, and wait. It should be done fairy quickly.

 

Code?:

Code:

import threading
import json
import urllib.request
def downloadFile(url):
	return urllib.request.urlopen(url).read()
def saveFile(content,location):
	fileStream=open(location,'wb')
	fileStream.write(content)
def downloadItems(type):
	print("Downloading:"+type)
	json_text=downloadFile("http://media1.clubpenguin.com/play/en/web_service/game_configs/paper_items.json").decode("utf-8")
	decoded=json.loads(json_text)
	for item in decoded:
		try:
			itemContent=downloadFile("http://media1.clubpenguin.com/play/v2/content/global/clothing/"+type+"/"+str(item['paper_item_id'])+".swf")
		except urllib.error.HTTPError:
			if type=="paper": #We want it output only once.
				print("Item "+str(item['paper_item_id'])+" is non-existent, skipping")
			itemContent="Fake&Gay"
		if itemContent!="Fake&Gay":
			saveFile(itemContent,type+"/"+str(item['paper_item_id'])+".swf")
			if type=="sprites": #Photos might be sad
				print("Downloaded item: "+str(item['paper_item_id']))
	
	print(type+" are DONE!")
sprites=threading.Thread(target=downloadItems,args=("sprites",))
paper=threading.Thread(target=downloadItems,args=("paper",))
icons=threading.Thread(target=downloadItems,args=("icons",))
print("""
Club Penguin Items Downloader
By albertvision
Made specifically for rile5!
""")
sprites.start()
paper.start()
icons.start()

  • Thomas, Mouse, Jordan and 2 others like this

#2
OFFLINE   TnT

TnT

    Administrator

  • Members
  • 1997 posts
  • Programming:PHP, MySQL, and Python.

User's Awards

        

Nice, but where will it save the items?



#3
OFFLINE   Jordan

Jordan

    Left.

  • Members
  • 1067 posts

Nice, but where will it save the items?

In the folders: sprites, icons and paper.

Great job albertvision! That's awesome!


  • albertvision and Wolfy like this

#4
OFFLINE   albertvision

albertvision

    User

  • Members
  • 1014 posts
  • Programming:PHP, MySQL, HTML, CSS, JavaScript, AJAX, jQuery

Nice, but where will it save the items?

In the current folder, in the example above items/



#5
OFFLINE   Mouse

Mouse

    Zeta

  • Banned
  • 7332 posts

newpath = r'sprites' 
if not os.path.exists(newpath): os.makedirs(newpath)

 

newpath = r'icons' 
if not os.path.exists(newpath): os.makedirs(newpath)

 

newpath = r'paper' 
if not os.path.exists(newpath): os.makedirs(newpath)

 

Nice script btw!



#6
OFFLINE   iWaddle

iWaddle

    Violett

  • Members
  • 1163 posts
  • Programming:PHP, Python, JS, AS2, C#

User's Awards

                    

Its the fastest? Dosen't it depend on the speed of your internet? Or does this make a difference to the speed it goes to the item and saves it? Oh well, thanks for releasing.



#7
OFFLINE   albertvision

albertvision

    User

  • Members
  • 1014 posts
  • Programming:PHP, MySQL, HTML, CSS, JavaScript, AJAX, jQuery

Its the fastest? Dosen't it depend on the speed of your internet? Or does this make a difference to the speed it goes to the item and saves it? Oh well, thanks for releasing.

It still depends on your internet speed, but instead of downloading one file, you download 3 at once.


  • Jordan likes this

#8
OFFLINE   Slider

Slider

    Zeta

  • Trusted
  • 4061 posts
  • Location:http:\/\/rile5.com\/
  • Programming:Python, Java

User's Awards

           

Would it download faster on a vps? and how do you install python on a vps?



#9
OFFLINE   albertvision

albertvision

    User

  • Members
  • 1014 posts
  • Programming:PHP, MySQL, HTML, CSS, JavaScript, AJAX, jQuery

Would it download faster on a vps? and how do you install python on a vps?

What OS are you running the VPS on?



#10
OFFLINE   Slider

Slider

    Zeta

  • Trusted
  • 4061 posts
  • Location:http:\/\/rile5.com\/
  • Programming:Python, Java

User's Awards

           

Ubuntu 12.04 its only a crappy host1free, but i'm only using to to create an iCPPS server(bored of opencp)



#11
OFFLINE   Banksy

Banksy

    Twerk Team

  • Trusted
  • 1562 posts
  • Location:Tumblradon
  • Programming:CSS, HTML, PHP, Javascript, AJAX, jQuery, SQL

User's Awards

        

Looks a lot better then mine. http://rile5.com/top...-downloader-21/



#12
OFFLINE   albertvision

albertvision

    User

  • Members
  • 1014 posts
  • Programming:PHP, MySQL, HTML, CSS, JavaScript, AJAX, jQuery

Ubuntu 12.04 its only a crappy host1free, but i'm only using to to create an iCPPS server(bored of opencp)

sudo apt-get install python3.3 or something similar.

 

Looks a lot better then mine. http://rile5.com/top...-downloader-21/

Suure... I totally have a design, and not just an output. Is your threaded?



#13
OFFLINE   Banksy

Banksy

    Twerk Team

  • Trusted
  • 1562 posts
  • Location:Tumblradon
  • Programming:CSS, HTML, PHP, Javascript, AJAX, jQuery, SQL

User's Awards

        

sudo apt-get install python3.3 or something similar.

 

Suure... I totally have a design, and not just an output. Is your threaded?

 

Mines ugly as hell... jk, jk. It is threaded.



#14
OFFLINE   albertvision

albertvision

    User

  • Members
  • 1014 posts
  • Programming:PHP, MySQL, HTML, CSS, JavaScript, AJAX, jQuery

Mines ugly as hell... jk, jk. It is threaded.

Well, mine is...

mine is...

mine is...

I FIGURED IT OUT!

Cross-platform. Heeh, Heeh.


  • Mouse and Banksy like this

#15
OFFLINE   Thorn

Thorn

    Gamma

  • Members
  • 728 posts

User's Awards

        

You made a little mistake / forgot to add it. The code you posted just downloads the paper swfs for everything.

Change this line:

ItemContent=downloadFile("http://media1.clubpenguin.com/play/v2/content/global/clothing/paper/"+str(item['paper_item_id'])+".swf")

to

ItemContent=downloadFile("http://media1.clubpenguin.com/play/v2/content/global/clothing/"+type+"/"+str(item['paper_item_id'])+".swf")

  • albertvision and Symfox like this

#16
OFFLINE   albertvision

albertvision

    User

  • Members
  • 1014 posts
  • Programming:PHP, MySQL, HTML, CSS, JavaScript, AJAX, jQuery

 

You made a little mistake / forgot to add it. The code you posted just downloads the paper swfs for everything.

Change this line:

ItemContent=downloadFile("http://media1.clubpenguin.com/play/v2/content/global/clothing/paper/"+str(item['paper_item_id'])+".swf")

to

ItemContent=downloadFile("http://media1.clubpenguin.com/play/v2/content/global/clothing/"+type+"/"+str(item['paper_item_id'])+".swf")

Thanks! I missed that one :(



#17
OFFLINE   Nobuddy

Nobuddy

    Beta

  • Members
  • 212 posts

Looks a lot better then mine. http://rile5.com/top...-downloader-21/

Is that sarcastic? if its not, Your is way better still.



#18
OFFLINE   albertvision

albertvision

    User

  • Members
  • 1014 posts
  • Programming:PHP, MySQL, HTML, CSS, JavaScript, AJAX, jQuery

Is that sarcastic? if its not, Your is way better still.

He was sarcastic about the looks, because mine is just a shell, but mine is multi platform and doesn't have to care about showing a GUI, so it might be a tiny bit faster



#19
OFFLINE   DJ_MuTeD

DJ_MuTeD

    Me.

  • Veterans
  • 1259 posts

User's Awards

           

shit + threading = BEST SHIT EVER






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users