How to extract all used hash160 addresses from Bitcoin blockchain

Question

I have all 150GB Bitcoin blocks now what? How to open them and read them in Python? I need to extract all used hash160 so far

I tried to open them with Berkeley DB but no success it seems these files aren't Berkeley DB and what is the difference between blkxxxxx.dat and revxxxxx.dat files anyway? it seems revxxxxx.dat files got some improvement in file size

Answer 1

you can use the python-bitcoin-blockchain-parser library. It allows you to parse Bitcoin blockchain data stored in the .dat files. The library provides an interface to access information from the blocks, transactions, and outputs.

pip install bitcoin-blockchain-parser

then Once you have installed the library, you can use the following code to extract the used hash160 addresses from the Bitcoin blocks:

from blockchain_parser.blockchain import Blockchain

blockchain = Blockchain("/path/to/bitcoin/blocks/folder")
hash160_addresses = set()

for block in blockchain.get_unordered_blocks():
    for tx in block.transactions:
        for output in tx.outputs:
            if output.addresses:
                for address in output.addresses:
                    hash160_addresses.add(address.address)

# Print all the hash160 addresses
for address in hash160_addresses:
    print(address)

When processing the Bitcoin blockchain, you generally work with the blkxxxxx.dat files to access the block data, while the revxxxxx.dat files are used internally by Bitcoin Core for efficient block lookup.

Keep in mind that working with the entire Bitcoin blockchain dataset can be resource-intensive due to its size. It may be helpful to test your code on a subset of blocks initially or consider using a database like Bitcoin Core's LevelDB or other specialized tools to handle the data more efficiently.