After downloading file we can see that it's x86 ELF. The description tells us nothing, so the best way to understand what is it --- execution:) When we execute it, we will see a newly created conlose with very-very fast changing pictures (every picture is a set of ASCII symbols). Hmm.. maybe something intresting is hidden in those pictures?.. Let's take a look!
A simple search of symbols from last picture produced no results, so they seems to be encrypted, gziped or saved in unusual way. To understend it let's load our executable to IDA and start reversing.
After fast research it's became clear that the greater part of this binary are compilated parts of different open source projects (e.g. OpenSSL sources has been used for cryptographical purposes). From one hand, it seems to be a problem: big projects have template-like structure and it's own data representation, what makes fast research very hard. From other hand, such projects have good error messages with functions or file names, and in many cases big projects with template-like structure provide a set of functions, which allow easy enumeration of elements of input array and making simular actions under every single element. The most common way to realize such functionality is the next:
typedef void (*with_element)(element &el); void for_each(array<element> arr, with_element foo);
According to this suggestion, the first thing we do is rearch for high-level function which enumerates pictures and write them to console. It's a function at address 0x0804ABB0:
Now everything we need is to set breakpoint at the end of function sub_804A910 or sub_804A460 and saw pictures. As practice has shown, it's better to set breakpoint at the end of function sub_804A460, because this function executes ones in conlose's pictures modification (e.g. at address 0x0804AA87).
The recived pictures are the next:
After some bruting (not all symbols can be easily recognised) we take a flag: "CTF+WINT3R+S3A50N". It's not very hard for 300 points:)