Sunday, July 4, 2010

Updates

Nothing terribly new, but I've updated ineptkey & ineptepub and unswindle.

For the INEPT tools I mostly just integrated and cleaned up the changes made by others. I did however (a) incorporate a version of the Mac OS X key-extraction logic into ineptkey; and (b) allow ineptepub to use OpenSSL instead of PyCrypto, removing the need to install PyCrypto on OS X. These two changes should greatly simplify ADEPT EPUB DRM removal under OS X. I elected not to integrate the "bulk decryptor" ineptepub changes -- my feeling is that anyone who needs bulk decryption can do it via the command-line.

For unswindle, I've just updated the tool to handle the most recently-released version of K4PC. If I missed a version or so in there, please try upgrading K4PC before complaining that it doesn't work. I am aware of skindle, but (a) one needs something like unswindle to run newer versions of K4PC under on Linux Wine anyway; and (b) I think the unswindle UI is nicer, as one can't identify a particular K4PC book just by filename.

Happy reading!

Update 2010-07-06 -- I had a bug in ineptepub which was causing segmentation faults on some versions of Mac OS X. It should be fixed for versions 5.2 and beyond.

79 comments:

  1. Hm, using the ineptepub script on OS X, Python suffers an unexpected error and has to close when I finally hit "Decrypt." The terminal lays the blame on a segmentation fault. Anything I can do to avoid this?

    ReplyDelete
  2. @Interrogative27 Can you report your versions of OS X, Python, and OpenSSL?

    ReplyDelete
  3. @Interrogative27 I notice one place where ineptepub could segfault if passed an invalid userkey. I added better error-checking and released it as version 5.1: http://pastie.org/1031306. Let me know if that helps.

    ReplyDelete
  4. I get a segfault with version 5.1 too.
    Python 2.6.1
    Mac OS X 10.6.4
    libcrypto 0.9.8 (comes with Mac OS X)

    ReplyDelete
  5. I can report exactly the same symptoms (ineptepub v5.1 seg faulting) on exactly the same hardware Mac OS 10.6.4, Python 2.6.1, openssl/libcrypto 0.9.8.

    One theory I have... it looks like I have multiple private keys in my activation.dat file. Does ineptkey and ineptepub corretly handle multiple certificates and keys associated with a single ADEPT activation.key file?

    ReplyDelete
  6. @Interrogative27 @V Ok, I managed to reproduce and fix the bug. Version 5.2 should have you good-to-go: http://pastie.org/1032773.

    ReplyDelete
  7. @YellowBear This issue was a straight-out bug in my code, but I'm not sure if does or not. Does ineptkey run cleanly? Please let me know if the 5.2 release of ineptepub has any issues with the extracted key.

    ReplyDelete
  8. New version of K4PC out, just saying. :)

    ReplyDelete
  9. @i♥cabbages Version 5.2 solved things for me as well. I'm still not sure why I have three XML blocks in my activations.dat file, all with apparently different private keys and certificates. I tried removing different blocks to see if it they changed what ineptkey.py did. The results was either no change at all or a python exception. Weird...

    I did run into one other problem. Whether it was deliberate or not, I can't say. But a book I purchased from fictionwise had a deliberately corrupted entry for the .opf file in the zip central directory. This cause ineptepub.py to throw an exception:

    zipfile.BadZipfile: File name in directory "Tale_9780456pg" id="f085" media-gg" and header "Tale_978045606921_epub_opf_r1.opf" differ.

    Adobe Digital Editions apparently only looks at the local/header filename, so it has no problem reading the file. I manually unziped it, and unzip trusts the central directory more than the local/header filename, but it displays both, and after I manually renamed the file and rezipped it, ineptepub.py was able to decrypt it. It could have been an accidental corruption that was never noticed because ADE doesn't pay attention to the zip file's directory --- or it could have been a deliberate attempt to make life difficult for people using, ah, non-standard tools to manipulate the file. I'm not sure which, but I thought I would mention it here.

    ReplyDelete
  10. Hi guys...
    I managed to get this to work on Win, but want to get it to work on my MAC... I don't get how to install pycrypto on OsX but the ineptepub works fine with me with OPEN SSL...is there a ineptpdf file that doesn't need pycrypto on OSX, but uses OPENSSL like the ineptepub...help would be appriciated...
    Thanks
    Laurent

    ReplyDelete
  11. @YellowBear I'm glad you're able to decrypt in the common case now. As for the BadZipFile exception, I'd just come across that looking over other comments from the past few months. If you could send a copy of the book with this problem (still DRMed) -- and ideally your adeptkey.der -- to i.u2665.cabbages@gmail.com, I can see if I can come up with a fix.

    ReplyDelete
  12. This comment has been removed by the author.

    ReplyDelete
  13. I'm using the new Mac method, but I'm encountering "File Name Errors" when I click the Decrypt button. It creates a file, but it's corrupt and so am unable to open it. An example message is: Error: File name in directory "OEBPS/shakey_22=
    Can anyone help getting around these errors? Is there a way to remove them or to have the script ignore them?

    ReplyDelete
  14. Am getting an error that says "
    import subasyncio
    ImportError: No module named subasyncio

    ReplyDelete
  15. @Spinoza I believe that's the ZIP file directory vs file header mismatch issue some other people are seeing. I'm working on a fix, but there's a work-around involving unziping and re-zipping the file -- you should be able to find instructions floating around.

    ReplyDelete
  16. @Jane Hmm. That is very confusing. I'm not sure what would be trying to import such a module. OS, version, Python version, etc?

    ReplyDelete
  17. Don't know where my question went, so excuse me if this is a double post. I renamed unswindle 7 and put it in Python26, where mobidedrm is located. I got this error message:
    Traceback (most recent call last):
    File "C:\Python26\unswindle.pyw", line 851, in gui_maininpath, pid = unswindler.get_book ()
    File "C:\Python26\unswindle.pyw", line 792 in get_bookgrabber = PC1KeyGrabber(debugger, self._hexdigest)
    File "C:\Python26\unswindle.pyw", line 652, in_init_debugger.set_bp(self._taddr(addr), getattr(self, mname))
    File "C:\Python26\unswindle.pyw", line 461, in set_bpbyte = self.read_process_memory(addr, type=ctypes.c_byte)
    File "C:\Python26\unswindle.pyw", line 451, in read_process_memoryraise DebuggerError ("could not read memory @ 0x%8x" % (addr,))
    DebuggerError: ciould not read memory @ 0x1000000000016591f

    I did it all over again logged in as administrator, and got the same error messsge. I am running Windows 7 and K4PC 1.1.1
    I am currently running skindle, so this is not an emergency, but I would like to try unswindle. I love ineptepub and ineptpdf. Thanks, Debra

    ReplyDelete
  18. Thanks, Cabbages, that did the trick, I was able to solve the problem by zipping and unzipping.

    Now, I'm trying to get the last of my DRM'd PDFs done, but when I run the new Mac-integrated key along with ineptpdf v. 7.4 I'm getting that "Error decrypting book session key". How do I circumvent this? Is there a fix for this? The best solution I found was that I needed to go back to DE 1.7.1. I tried this but I still got the same error. Any help here would be appreciated!

    ReplyDelete
  19. The fastest way to fix the broken ZIP central directory is to do "zip -FF foo.epub --out foo_fixed.epub"

    ReplyDelete
  20. Any chance to remove the VitalSource DRM on VBK ebooks?

    That would be awesome!

    ReplyDelete
  21. Any chance of putting the latest scripts on pastebin, as pastie seems to be down at the moment :(

    ReplyDelete
  22. If you had an epub format file that was AES 128 bit encrypted but not processed through with RSA (there is no rights.xml file), would your ineptepub code be able to be modified to decrypt said file?

    I see you use the rights.xml's encryptedKey to create the "bookkey" variable and populate it. I was wondering if the epub has never gone through the complete process, would your code be able to work? I guess the short question is: can you decrypt AES based solely off the KeyInfo and Resource information in encryption.xml?

    ReplyDelete
  23. Thanks a million for these scripts! Really, awesome work.

    ReplyDelete
  24. There's a new version of K4PC out. Have you had a look at it?

    ReplyDelete
  25. There's a new version of K4PC, but enyone can dovnload older version of that program and disable automatic update.
    I had other problem. Unswindle dosen't support polish letters in file path name (I had letter "ł"). I had to change my windows account. My previous one was "Ł**** M******".
    Anyway its great program and now i can read my books the way I want. Thank You verry much :).

    ReplyDelete
  26. I am trying to run this on XP but I keep getting an message: Error:Failed to determine book path. I have disabled the firewall and the UAC. I don't know where to look next. Thanks in advance.

    ReplyDelete
  27. Just tried unswindle, reported that it does not support the version of K4PC I have: 1.2.0 (30413). Any chance of an update to unswindle?

    ReplyDelete
  28. Just tried unswindle, and complains that it doesn't support my K4PC version, which is 1.2.1 (30427). Please update unswindle :)

    ReplyDelete
  29. Very impressed - are you still updating this software, though? Unswindle being the main example

    ReplyDelete
  30. Hi,
    Thanks for your job. Do you have something new regarding the "Not an Adept Epub", when there is no "rights.xml" inside the archive ?

    ReplyDelete
  31. I have read through all of the blog posts and I need some help. I have Python 2.6.6 installed along with the adobe digital editions. I have the "Keys" open on other tabs and also downloaded but I'm at a loss in regards to what I should do next. Can anyone help me with simple explanations of what to do, I'm pretty new at this.
    mauimom5@gmail.com

    ReplyDelete
  32. Help! I'm not a code writer nor know how. I just want to read the DRM'd books I ended up getting from B&N in Stanza. I'm having a tough time figuring out how to work the files you did. Can you assist a neophyte like me?
    Am on a Mac with 10.5.8 with an iPod using Stanza.

    Patrick

    ReplyDelete
  33. Trying to de-fang a NookStudy (pdf) book that I bought so I can put it on my Nook (crazy, yes?): Getting the following error: unsupported security handler `EBX_HANDLER'.
    FlateDecode: decoding error: incorrect header check.
    FlateDecode: decoding error: incorrect header check.

    Anyone else have similar problems?

    ReplyDelete
  34. Hi IHeartCabbages,

    People have made slight modifications to your ineptepub, ineptkey, ignobleepub, ignoblekeygen, to allow use of OpenSSL on all platforms to remove the need for PyCrypto completely. It will work for Windows if they install OpenSSL, and for Mac OS X and Linux out of the box. People also have found and fixed a minor bug with OpenSSL librcrypto 0.9.8 before patch-level o that could result in segfaults.

    If you do plan on updating your scripts, please consider incorporating these changes as well.

    The revised scripts are available as part of a tools_v2.2a.zip package that can be downloaded from here:

    tools_v2.2a.zip
    http://www.mediafire.com/?8mzrezba1bvmuqd

    Thanks.

    ReplyDelete
  35. This comment has been removed by the author.

    ReplyDelete
  36. i♥cabbages: What should I do if I get the "Not an ADEPT ePub" error?

    I downloaded a book from Borders.com, and I'm pretty sure all of their eBooks are have ADEPT DRM.

    If the book isn't ePub standards compliant, or something like that, is there a way to fix the file before being processed by your script?

    ReplyDelete
  37. If you get the "No an ADEPT EPUB" error, it probably means you downloaded the book incorrectly from the publisher.

    At least from Borders.com, you get a .acsm file when you ask to download your ePub book file, but the .epub file mentioned in that .acsm file is missing the rights.xml file inside the archive.

    You have to open the .acsm file in "Adobe Digital Editions", which is Adobe's shitty book reading program supported only in Windows and MacOS, that reads stupid Adobe ADEPT encrypted ePub files.

    Once you open the .acsm file with Digital Editions, you will find a .epub file in "My Documents/Digital Editions" which will contain rights.xml, and can be processed with i♥cabbages' script.

    Go here for more details:
    http://superuser.com/questions/208587/remove-drm-from-epub-files

    ReplyDelete
  38. I'm having the same troubles as Neil. I purchased the book from borders and extracted the ePub file from the Digital Editions folder. I finally got the key script to run on my Mac, but I get another error that says "unexpected error". Last line says "ExpatError: no element found: line 1, column 0.

    I did extract the ePub file from "my documents/Digital Editions" and have it in the same directory as the scripts, so I don't understand what I'm doing wrong. I tried deleting and re-downloading the ePub with the acsm file, but I still get the same error. I also tried putting the scripts in the Digital Editions folder to see if they would run with the ePub in there, but that also didn't work.

    I tried opening the same scripts in Parallels with Windows XP and Python and PyCrypto installed. And again, I get the No ADEPT file error.

    The ineptpub file runs just fine, but it doesn't do me any good unless I can get the key script to run.

    Any ideas?

    ReplyDelete
  39. I've been converting my Kindle books to epub format so I can read them on my iPad using iBooks. Some of them are in the older AZW format. The tools v2.2 work fine in converting, except they drop most formatting - like italics, underline, etc. Is there any way to retain the formatting, or modify the scripts to retain it? I'm using TopazExtract, TopazFiles2SVG, and TopazFiles2HTML. Thanks, and great work!!

    ReplyDelete
  40. I keep getting either:
    The AES file must be 16 32 ....
    or
    The AES key cannot be a null string
    Does this have anything to do with AMEX being a 15 digit CC

    ReplyDelete
  41. Kevin's toolkit mods worked!!! Many, many many thanks
    jc

    ReplyDelete
  42. Help!

    I'm an avid E-Book reader with my iPod and am having a real hard time dealing with those pesky DRM's. I found Stanza to be the best and Goodreader for pdf's.
    What I am baffled by is getting Calibre to work right so I can read those E-books I've purchased from a variety of vendors with my Apps of choice.

    I would be most thrilled and incredibly appreciative if someone, anyone, could contact me off-list with their phone number and and appropriate time to call who I can have a walk-through to get this working on my Mac.

    I've been frustrated by many of the terms and descriptions used as well as the various files and their uses and been sent down several off roads that just caused my sight to blur. Exasperated.

    Anyone, I need a Talk-through. I'm not dense, just not familiar with the confusing terms everyone uses.

    patrickforbesatmacdotcom

    ReplyDelete
  43. any ideas on how to unprotect Lizard Lock ebooks. I hate that I cannot read them on my nook color

    ReplyDelete
  44. This comment has been removed by the author.

    ReplyDelete
  45. Hi:

    In case u wonder how folks use your scripts--I'm blind, & apps put out by Barnes & Noble, Adobe, &, up until just a few days ago, Kindle, are completely inaccessible to screenreading technologies. Because of your scripts, I'm able to buy & read books or borrow them from a digital library just like my sighted counterparts. Since by law in the United states blind people are in fact given permission to circumvent encryption technology that keeps us from reading, it's actually legal for me to do this. So thanks for helping. It's really unfair that materials which could be made accessible to us & which could broaden our reading possibilities are so restricted. Seems like most sighted folks think that everything's automatically translated into Braille, but the truth is that between talking books & braille, we only get about 3000 books a year, which is about the number of books that's put out in < a month. So you're really helping a very underserved segment of the population, though u probably didn't intend it so. Thanks again.

    ReplyDelete
  46. Do people know the difference between being biracial, multiracial, Creole, and fair skinned?

    thx
    Brad Fallon

    ReplyDelete
  47. Hi,
    Nice work!!
    I have updated K4PC and now unswindle doesn't work anymore :(
    Any update in sight?
    Thanks

    ReplyDelete
  48. I posted on the old blog but will repeat here. Apparently there is some new form of DRM that B&N has implemented.

    I've only been able to unencode about 30% of the books I've bought from B&N with the new Python scripts.

    Anyone notice this?

    ReplyDelete
  49. It mentions bulk can be done from command-line, but how? - I don't know python(basically none) enough to hack in the feature to do GUI bulk but i am otherwise quite computer literate...

    ReplyDelete
  50. My wife and I have different Ereaders, different sources for books, but one Win 7 computer with separate profiles. She has her own ADE account also. I did the initial research on ebooks, downloading ADE, downloading Calibre, Python, and other tools. I have been successful with the Calibre Plugins and creating the adeptkey.der file using the scripts but only under MY Windows Profile.

    Under her Profile I cannot create the key file, It says: "Error: Failed to decrypt user key key (sic). When I try to remove the DRM from EPub adept files with the drag & drop method, no luck. And also when adding into Calibre with the scripts, they are added but DRM is not removed.

    Is it possible to be able to have two user profiles with two key files with these tools? How to make it happen? I have given her profile admin rights.

    ReplyDelete
  51. Not totally certain what Rob fixed.

    I was able to work around my problem by installing Python, PYCrypto, ADE on another computer (XP), authenticate my wife's ADE account on that computer, create the certificate, then copy the certificate file to our main computer.

    Able to use Calibre and ineptepub tools to anti DRM her books and mine. Have not tested the drag & drop tool.

    ReplyDelete
  52. Bruce, Rod or anyone else:
    As a Mac user, but nowhere near a 'Guru', this Calibre thing has me VERY confused. I've stopped buying E-books as I can't get Stanza to open them up.
    I could really do with some one-on-one assistance to help me get this ironed out. Anyone?

    ReplyDelete
  53. Thank you so much for your great work. My mother bought an Kindle, and then she bought a lot of DRMed Epubs... You saved our day!

    ReplyDelete
  54. I am getting what seems to be the Unzip error, but I'm a little confused because I don't have an uzip file. I'm on a Mac and have a .acsm file which is an XML file that ADE uses to download the actual content. When I try to decrpyt it I get an error about a file name difference in the header. I see the zip file work around discussed, but I don't have a zip file to work with. Any suggestions on this?

    Thank.

    ReplyDelete
  55. @HarryO!

    HarryO is right, new B+N pdfs are decrypted by the program successfully (ie. successful decryption appears), but the files are corrupted.

    Some pdfs work, some dont

    ReplyDelete
  56. Can ineptkey be modified to work in linux
    (without cheating using wine)

    ReplyDelete
  57. I am also getting the "Error: Failed to decrypt user key key (sic)." It's Wine under Linux, which may be a problem, but I have seen other users reporting successes.

    I'm happy to do the debugging if I knew where to start - suggestions?

    ReplyDelete
  58. Today I bought an ebook for the first time (from Blackwells). As of this morning, I had never before heard of ADEPT and only vaguely heard of DRM (I use Ubuntu). I was shocked and annoyed that I couldn't even download my ebook using Ubuntu (no Adobe Digital Editions).

    Yeah, OK, go ahead and laugh....

    So, after a bit of googling... find an old XP, download python and your scripts and smooth as silk, now have a ebook that I can read on my Nook! Fantastic.

    You sir, are a star.

    ReplyDelete
  59. Is it necessary to remove earlier updates downloaded for various programmes and keep the latest one only?
    2nd birthday party themes

    ReplyDelete
  60. This comment has been removed by the author.

    ReplyDelete
  61. This comment has been removed by the author.

    ReplyDelete
  62. Updates are something really essential. Thanks for sharing this information cause it will be very useful for me!Thanks!

    ReplyDelete
  63. Updates are the foundation of good working process in my opinion! I absolutely agree with everything you said in the article! Clean House

    ReplyDelete
  64. How do I update my Facebook status from my cell phone without having other peoples updates texted to my phone?

    eb-5 visa

    ReplyDelete
  65. running python 2.6 with working pycrypto on 64b windows 7. I get the key okay and then when I try to decode I get the not an ADEPT EPUB error. Tried several different files, a temp and the original but nothing works. Just using this as a way to teach myself more about python et allia, am I missing something obvious or what??? I read and tried the comments from Niel above.

    Book is from overdrive, new book, just want to see if I can move it around from reader to reader. Thanks in advance for any suggestions.

    ReplyDelete
  66. This comment has been removed by the author.

    ReplyDelete
  67. Thank you for an extra good writing. What are the local people can get in such a perfect way to write the details? I have a presentation next week, and I look around this information and facts. gem stone jewelry

    ReplyDelete
  68. Hi,
    Hope I am posting this in an acceptable place. The Readme file just said "post a comment in your blog . . ."

    Got a new computer and had to reinstall Calibre and DRM removal plugins, which had been working just fine on my old machine. Now the DRM is not being removed when I add an Adobe epub to Calibre. I followed the instructions to run the process from a command line and the results were as shown below. Hope you can help.

    Microsoft Windows [Version 6.1.7601]
    Copyright (c) 2009 Microsoft Corporation. All rights reserved.


    C:\Users\ldfs>calibredb add "C:\Users\ldfs\Documents\My Digital Editions\Nanjing
    _Requiem.epub"
    IneptEpub: Using pycrypto.
    IneptEpub: Calibre configuration directory = C:\Users\ldfs\AppData\Roaming\calib
    re
    IneptEpub: Couldn't Retrieve key from ADE install.
    Running file type plugin Inept Epub DeDRM failed with traceback:
    Traceback (most recent call last):
    File "site-packages\calibre\customize\ui.py", line 148, in _run_filetype_plugi
    ns
    File "calibre_plugins.ineptepub.__init__", line 455, in run
    ADEPTError: IneptEpub - No keys found. Check keyfile(s)/ADE install
    Backing up metadata
    Added book ids: 7
    Notifying calibre of the change

    C:\Users\ldfs>

    ReplyDelete
  69. hello hello
    i have been hunting every where 2 pdf questions. i heard u were a pdf guru.....

    so i post here, in big hope you have some idea.....

    1) is it possible to create a pdf (whatever tool) where the images retain as images e.g. I long tap the image on my iphone (say in email or goodreader app) and i get the same popup as i do when i long hold the image in email. that is the iphone asks you "do you want to same image to camera roll"

    no matter what pdf tool i use, it seems images "as images" are gone. they like get flattened or something.


    2) is there a way to keep links as hyperlinks in pdf creation?

    i have many pdf creation tools and cannot seem to do it. I know it is possible as i get many of my mobi and epub converted to pdf using the great online service ebook.online-convert.com, they retain the links.

    ive hunted for a pdf mechanics book too. Ive read PDF hack but no luck. if you have any other recommendation, be great to receive. thankyou

    thankyou in advance

    ReplyDelete
  70. Hi ...
    I've been following your blog for years, know you're "the DRM guru" and hope to ask for some help with decrypting the B&N PDF files (PagePerfect Nook Books) - no one has been able to crack the scheme so we're tied to using the NookStudy reader (which leaves much to be desired). This all seems to have started in June 2013 when the B&N folks dropped support for Nook4PC and began directing people to NookStudy. The Nook4PC won't download /display the new B&N books. So we're stuck with these lousy encrypted PDFs.
    Being in Canada, I can't even BUY a Nook...so I can't view the files on a Nook nor can I convert the stupid files to read them on another e-reader. I'm begging for help...

    I'd be grateful for any assistance!!

    Thanks in advance...
    str8arrow

    ReplyDelete
  71. This comment has been removed by the author.

    ReplyDelete
  72. Dear i♥cabbages,

    I wonder if you have any thoughts on the new ADE3/RMSDK10 "Hardened DRM" that Adobe is rolling out over the next few months?

    Many thanks for your wonderful work on Adobe's "Adept" DRM scheme.

    ReplyDelete
  73. Thanks for your work. I wonder if you've ever had a chance to look at Audible DRM? It seems far better obfuscated, but the underlying format is AAC, which is of course widely supported. If the DRM could be stripped, it would make a lot of audiobook listeners much happier!

    ReplyDelete


  74. Banner Design Service is a focusing fact for any kinds of business institutions. Banner show your products identity which is provided by you. Manage your banner design service creatively by any professional designer to highlight your products. Creative banner grabs your customer’s attention.


    ReplyDelete
  75. I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often. 백링크

    ReplyDelete