How to edit an old blog post so that it looks like a prediction...

In a world where fake news seems to be a major part of the news, then it is interesting to see just how easy it is to break the trust that people put into 'systems' and their senses. '

For senses, then 'I only trust what I can touch with my own hands, or see with my own eyes' is one example, which counterfeit goods, photoshopped images and 'deep fake' videos show isn't a very reliable way to assess if something is genuine.

For systems, then a time-stamped published blog post seems like it might be a modern digital equivalent of the classic 'Photo of a newspaper fixes the earliest possible date when the photo could have been taken...' scenario. So a blog post, which is stamped with the time and date that it as published, might seem to be a good way of showing when you first published a thought, idea or comment.

Unfortunately, the design of many systems is not perfect, and sometimes it doesn't do what it appears to do. Blog posts, for instance. The time and date that are shown in Google Blogger (which is what I use to publish this blog) are when it was first published. Any changes after that do not change the time or date, because a blog (from 'web log') is meant to be a series of 'diary'-like entries, and you don't generally edit your diary... So the design of a blogging application (or program, as they used to be called!) has a time-stamp for the publishing date as a key requirement, but there's no requirement at all for time-stamping any edits, and in fact, if you did change the time-stamp for each edit, then it would stop being a log. Even worse, suppose that a picture, photo, graphic, web-site, web-page, or an article in the published blog post was replaced or updated (the original disappeared, for instance), then changing the publishing date changes the time and date of the blog even though none of the major part of the text has changed. What happens when a different advert is placed in the blog post?

So, by design, the time-stamping in Google Blogger (and many other blogs) is a useful way to find out when a blog post  was first published. But that is all. Any subsequent edits are probably not reflected in the 'published on' time and date stamp.

A security-minded person looks at this design and sees a flaw. Most people will look at the 'published on' time and date stamp and assume that it means when the blog post was published. The analogy with the time and date printed at the top of a newspaper is firmly locked in many people's minds. Even if edits were time-stamped, then how do you know you can trust the time-stamping process? Winding back the date on a computer so that '30-day' trials of software continue to work is a very old approach - and triggers an interesting 'vulnerability/mitigation' escalation 'ladder' if you try to stop it happening. These things boil down to: "How much time and effort is it worth to you, trying to make this perfect?', because whatever you do to try and secure your time-stamp will probably introduce one or more new possibilities for subverting it, albeit with more required effort. And nothing is perfect!

So, if you look at this blog post, from the 2nd of October 2018, you will see an edit that I made today to a blog post from more than 2 years ago... but the published date and time were not affected. As you can see, it looks like I had a bad feeling about 2020 way back in 2018 - or maybe I didn't and I just edited the blog post. Does this prove anything? Well, it proves this:

Don't trust blog posts - except blog posts that tell you not to trust blog posts!    

So editing is easy! And a little bit of 'thinking ahead' provides an interesting principle: if you publish a blog post a few times every month for a few years, then you can go back at any time in the future and edit it to say anything at all! I'm now wondering what I should predict next...

The Catch!

This wouldn't be a security blog post if there wasn't a 'gotcha'! Yep, whilst Google Blogger (or other blog apps) display the time-stamp for when the post is published, there are ways to find out when it was altered as well. The Internet 'Wayback Machine' grabs web-pages (Only 439 billion or so - not all of them!) and so can be used as a 'view into the past' - but it also allows pretty detailed investigations of when something has been changed. Now hacking the Wayback Machine is a possibility to cover tracks, but...

This is probably a good moment to remind you that useful resources like the Wayback Machine need money, so I encourage you to go to the web-page and donate! I have donated!

---

Whilst you are thinking about donating to the Internet Wayback Machine, then if you also find my writing helpful, informative or entertaining, then please consider visiting the following link for my Synthesizerwriter alias (I write several blogs, but it makes sense to only have one 'Coffee' donation link!):

Buy me a coffee

The considered view is better than the initial reaction...

Some time ago, when lockdown first started, there was a lot of mainstream media coverage about how insecure some videoconferencing apps were. As always happens these days, some security companies may have been tempted to use this as a way to get publicity by releasing reports detailing their investigation into the risks of using those videoconferencing apps, plus some videoconferencing marketing people might have considered using this as an opportunity to promote their product, and all of this was then reported by mainstream media with a variety of biases and hidden agendas, plus the ongoing desire to capture eyeballs and clicks. I've been intrigued for a while by the view that says that 'Fake News' is a new phenomenon, because I have always thought of all news 'information' as being potentially flawed and requiring a critical appraisal. More broadly, the:

'Trust no source, check everything'

approach has always been very useful insurance. One example that I'm familiar with is that some editions of some of the standard text-books on analogue filter design have contained errors in some of the formulas (or formulae). The tricky bit here is 'some', because this means that the usual 

'check in a couple of reference works' 

approach can fail, because they might both contain the same error! Also, in an online world, what counts as a 'reference work' these days? I have always been intrigued by the way that YouTube and other online social media platforms use words like 'authoritative' when describing sources of information - they don't use words like 'legal' or 'experts' or 'scientists' or 'legislators'. Now appearing to be 'authoritative' would seem to be rather subjective to me, whereas acquiring 'expert' status can be objectively assessed, albeit with caveats because the assessment can be flawed. 'Edition n contains errors, whilst edition n+1 fixes previous errors, but may still contain a different set of errors' is one way of looking at it.

When the mainstream media report on security, then there is a spectrum of opinions from security practitioners about how well they do it, ranging from 'They don't understand', through 'Most of this is kind of true, but...' to 'They understand'. My usual reaction is something like: 'They are describing some of the basic levels of this, but many of the important nuances and fine detail are missing, because to simplify a complex subject for a general audience is obviously a challenge.'

On Twitter I said that my first source of news was from inside the security community, and that Bruce Schneier or Brian Krebs (or Matthew Green, but I kept the Tweet short) would be my preferred way to get an initial informed view on any security issue that the mainstream media were talking about. And yes, I'm well aware that it is rare for the mainstream media to talk about security, and that the time delays in publishing specialised blogs and mainstream news are different. And no, the order wasn't meant to be significant!

So here we are some time after lockdown started, and this is probably a good time to look and see what the 'considered' opinions are.

Here's Bruce Schneier giving some thoughts, which refers to an NSA survey and another one from Mozilla, plus another from Matthew Green, and some on a specific app from Brian Krebs ... Many security companies and organisations have published guides on 'Things to Consider' when using a videoconferencing app or 'working at home'. - here are some examples from Kaspersky , ITGovernance , the New Zealand NCSC , and a the UK NCSC  ...

It is very interesting to take the surveys and to compare them to a lot of the mainstream media headlines, articles and some social media 'statements' that appeared in the first days of the lockdown. What you find is, and I'm repeating this deliberately: 'They are describing some of the basic levels of this, but many of the important nuances and fine detail are missing, because to simplify a complex subject for a general audience is obviously a challenge.' For me, it is interesting to see how many of the statements like  'X does end-to-end encryption (or choose your own feature of interest), Y does not.' are not backed up by the surveys - either as 'X and Y do not', or more interestingly and relevantly: 'it isn't as simple as that...'.

So is the considered view better than the initial reaction? I would guardedly say: 'Yes', but that's not a complete and definitive 'yes'. It depends...and that opens up a whole series of interesting things to explore about truth...

---

If you find my writing helpful, informative or entertaining, then please consider visiting the following  link for my Synthesizerwriter alias (I write several blogs, but it makes sense to only have one 'Coffee' donation link!):

Buy me a coffee

One day there's going to be an automated security disclosure...

I was just a little surprised when Instagram sent me a message that appeared to 'express an opinion'...

But it set me thinking about how easy it would be for a developer to write an automated message template that leaks confidential information via an unexpected side-channel... Risk assessment of tiny scripts that do apparently innocuous things, anyone?

Names are interesting things. I once tried to get a hi-tech music themed sticker printed by one of the on-line drop-shipping companies and the artwork was rejected because of a copyright strike. I had used the word: 'Device', as in an electronic music device, but this was flagged up because 'Device' was the name of an American industrial metal band in 2012, and so was trademarked...

---

If you find my writing helpful, informative or entertaining, then please consider visiting the following  link for my Synthesizerwriter alias (I write several blogs, but it makes sense to only have one 'Coffee' donation link!):

Buy me a coffee

Hardware.IO - my tiny bit of a major Virtual Conference! aka 'What are Wall Challenges'?

I was busy during April 2020. Several deadlines all conspired to converge on the same 'month end' delivery date. But one of them was the sort of project that I really like: puzzles!

Antriksh at Hardware.IO asked me if I could reprise the 'Wall Challenges' that I did for the 2017 and 2018 Hardwear.IO conferences in Den Haag (The Hague) in The Netherlands. '21 mysterious A4 pages blu-tacked to the walls with only a brief explanation' is something I've been doing for various events for some time, and it is a low-key, often mostly overlooked facet of the whole event - except for the people who get into it. If you go to one of the big, serious, 'suits' events, then you find teams of people who turn up just for the 'Capture the Flag' penetration or 'Capture the Signal' radio competitions etc., and they are usually pretty totally focussed on that for the whole event.

My 'Wall Challenges' are several opposites at the same time: they are carefully crafted training exercises, just like a 'Capture the Flag' contest; but they are also deliberately abstracted, which isn't a CTF or CTS feature. They are also fun challenges! So I thought that this might be a good time to look at what they are for, and why you might like to consider immersing yourself in one next time you see a sheet of paper stuck on the wall...(physical or virtual!).

Wall Challenges - what are they?

If you've ever wondered how people acquire an assured, casual ease with some technical subjects: 'facility' is one of the words that sometimes gets used, then one way to do it is not to read lots, or watch videos or attend lectures/seminars. Instead, actually doing something is often another good way to build familiarity, explore the limits of what you know (or don't know), and maybe extend your boundaries a bit. This is your cue: keep reading and do some WallChallenges!

Wall Challenges are apparently simple problems that are often harder than they appear, and doing them is good for you! If you ever wanted a Sudoku that was more than just a few numbers in squares, or that required programming to solve, or that went into the mathematics or theory a bit more, then you might find that Wall Challenges are exactly what you are looking for.

What sort of topics are covered? Things like Binary, Hex, Number bases, ASCII, ROT-13, Hashes, Look-up tables, Modulo arithmetic, Pointers, Pictograms, Anagrams, Cryptic clues, Codes, the Periodic Table, Lateral thinking, Critical thinking, and more. Solving them can often be done with just pen and paper, although some require a spreadsheet, and the harder ones can require some programming (Python is what I've used...). In the course of finding solutions, you will also acquire a collection of interesting look-up tables (ASCII, Periodic Table...) that often have interesting histories and are very good things to know for Pub Quizzes or Only Connect (My Team didn't get onto TV, by the way...).

There's a school of thought that making things simple is both a work of genius and a genius-level way of making it look trivial. When Einstein wrote 'E=mc squared' then it looked simple enough, but the ramifications were universe-altering. Now, Wall Challenges aren't quite at that level, but they can change the way that you think...and that's the whole point. These aren't trivial ways to pass time, they are intended to make you think about things that might well be useful in Penetration Testing, Risk Assessments, Threat Modelling, Security Analysis, White and Black-hat Hacking, and so on and so on.

I'm going to show examples, and in each and every case, the answer will be 'Cryptography', and it will be in capitals, which is supposedly how 'real' cryptographers do writing (sometimes). Not the crypto of block-chain currencies, but the 'hidden writing' of encryption, AES, CIA and several other three-letter acronyms (that's Confidentiality, Integrity and Availability), of course...). The Wall Challenges shown here are deliberately simple and easy to solve, plus you already know the answer! Real Wall Challenges are a bit harder, and some, like the ones you find at hardware security conferences like Hardwear.io, are very hard indeed.

Anyway,  here's the first Wall Challenge, which is called 'Bounce':

YCHRPYAPRTGO

Okay, so even though I've given you the answer, you are probably struggling to see how we went from cryptography to that! What you need are some strategies to get you started, and the first of these is:

Strategy 1: Start at the ends and scan across, skipping to see if anything looks interesting.

Well, the first letter is 'Y', reading from left-to-right, and 'Y' is the last letter of CRYPTOGRAPHY, but continuing gives 'YCHRPY' which isn't CRYPTOGRAPHY backwards (which is YHPARGOTPYRC, of course). The other end is 'O', and going right-to-left across gives 'OGTRPA' which isn't helping much either. 

So let's repeat that, but skipping every other letter, and we get 'YHPARG' and 'OTPYRC'. Woah! That's CRYPTOGRAPHY backwards isn't it? So if we start at the C, second letter in from the left, and miss out every other letter, then we get 'CRYPTO' as we go across from left-to-right, and then we need to reverse direction and go right-to-left to get 'GRAPHY'. So at the end of the word, we 'bounce' and reverse direction. Maybe there should be a brick wall graphic on the piece of paper on the right hand-side? 

So, we now know that doing a bit of adjusting of the order of letters can hide a word, but what use is that? Well, one of the basic transformations that encryption algorithms like AES use is shuffling the order of the data bytes...

Here's the second Wall Challenge, which is called 'Inside out':

PARG

HRCO

YYPT

Starting at the 'P' on the left and going across to the right doesn't give anything useful (PARG might be the start of PARGETER, but it is an unusual word, and there aren't any 'E's!), so try right-to-left from the 'G' - which gives 'GRAP' and we know that is in the middle of the word we are looking for... But in a real challenge then we would not know what the hidden word is, and so this wouldn't be that useful. 

What we probably need is a revised strategy:

Strategy 1: Start at the ends and scan across, vertically and diagonally, skipping to see if anything looks interesting.

If we do this from the lower left hand 'Y', then we get 'YHP', and if we turn the corner at the 'P' to go across to the right, then we get 'YHPARG', which is the end half of CRYPTOGRAPHY, but reversed. If we carry on going round then we eventually hit the 'Y' where we started, so let's turn and carry on in a spiral, which takes us all the way to the 'C', giving 'YHPARGOTPYRC', which is CRYPTOGRAPHY backwards again. So this time, the word was written 'inside out', as a spiral from the initial letter 'C'. Here's me trying to make it more obvious by using coloured letters for the first three letters in the spiral:

PARG

HRCO

YYPT

...and then the last letters...

PARG

HRCO

YYPT

What have we learned this time? Well, it seems that people who are used to reading from left-to-right can find it difficult to go from right-to-left, and that turning things into a 4x3 grid and using a spiral is hard to read. So the shuffling that encryption algorithms carry out looks like it can be effective at obfuscating (a fancy way of saying 'concealing') the sequence of letters in a word. Now, I'm not aware of any cryptographic algorithms that use spirals - they tend to just shuffle or rotate rows or columns. But spirals occur all over the place in nature, and people like them, so there may well be a bias in my usage of them.

The third challenge changes tack, and goes for numbers instead of letters, and is called 'Index':

3 18 25 16 20 15 7 18 1 16 8 25

Whenever numbers appear in a Wall Challenge, then you use another strategy:

Strategy 2: Are the numbers in decimal, hex or another base?

In this case, the numbers appear to be decimal. Often the '3' will be shown as '03' to make you think that it might be in hexadecimal or some other base. Notations like 0x8E for indicating hexadecimal numbers are quite rightly used in programming to make it unambiguously perfectly clear that the '8E' is in hex, but in Wall Challenges there are no rules, and so clues like '0x' are rare. In fact, if I did use that notation, then it would probably be mis-direction!

Oh, nearly forgot:

Strategy 0: There are no rules, standard practices or conventions. (The bad guys break them all the time anyway.)

So we have a list of numbers which might be decimal, so what do we do next? A variation of Strategy 1 is a good starting point: look at the ends, and then scan across and find the largest and smallest numbers. In this case, 3 and 25 are the ends, 1 is the smallest value, and 25 is the largest value. This information is full of clues - can you think of something that comes in a set with about 25 different members?

How about the alphabet? 26 letters... So starting on the left, what is the 3rd letter of the alphabet? 'C'. The 18th? Er, and here you get to the first lookup table. Open your favourite spreadsheet of choice and create a table that has the numbers from 1 to 26 in the first column, and then the letters from A-Z in the second column. Voila - you now have a useful Wall Challenge solving aid, and the beginnings of a collection of tables about symbols and numbers that you will be using a lot. Here's what I produced:

Producing lookup tables like this also has a strategy:

Strategy 3: Whenever you need a look-up table, make one, save it, and add a few extra columns so you are better prepared for next time...

For this table, I added the third column, which is a reversed index to the alphabet. This is good preparation for what myself and lots of other security analysts call 'The T-Shirt Effect'. We all wish that we had a T-Shirt that says on it: 'There's no way that would ever happen!', because this occurs  every time in a Risk Assessment or Threat Modelling session - there's always someone who says these words. In fact, governments around the world probably heard the same or similar words when they looked at the risk of a problem with a new virus epidemic at any time in the last decade...

Anyway, the table makes looking up the 18th letter of the alphabet much easier: 'R'. Going across from left to right, converting from the index number to the corresponding letter of the alphabet, we get: 'CRYPTOGRAPHY' just like you knew we would.

The fourth introductory Wall Challenge is a bit different, and is called 'Standard Interchange':

67 82 89 80 84 79

71 82 65 80 72 89

This time, the ends are bigger than the previous example, at which point experienced Wall Challenge solvers use another strategy:

Strategy 4: Is the range of numbers 26, 36, or some other small number that might contain an alphabet and numbers?

The lowest is 65, and the highest is 89 which is a range of 24, so immediately you should be suspecting something based on the alphabet. Now 65 is one of those 'magic' numbers that shouts out for attention, because the capital letter A is 65 in ASCII, the 'American Standard Code for Information Interchange'. 89 is Y, and so it looks like this is the time to get or make an ASCII lookup table for your collection.

If you replace 67 with the ASCII letter equivalent, you get 'C'. 82 is R, 89 is Y, and before you know it, you have: 'CRYPTOGRAPHY'.

There's an interesting thing to note here. The index table for the alphabet is actually a sub-set of part of the ASCII table - if you add 64 to the first column then it decodes CYRPTOGRAPHY perfectly fine, and this could be added as a fourth 'Shifted ASCII column'... But the ASCII table has lots of other characters in it - numbers, lowercase letters and all sorts of symbols, plus characters that control what a printer does (line feed, carriage return and those curious references back to mechanical printers that borrowed terminology and actions from mechanical typewriters...), as well as characters that don't actually print anything. If you go beyond the 127th character, then ASCII changes from something which is pretty consistent everywhere, to something with lots of alternatives. This 'Extended ASCII' is still standard, it's just that there are lots of standards covering all the variations.

So a nefarious puzzle-setter who wanted to hide some text might well make the capital A have the value 65, but that doesn't mean that it is automatically ASCII. Suppose B was 64, and C was 63? The correct reaction at this point is to already have your spreadsheet open and be adding a column, by the way...

That completes this first introduction to Wall Challenges. If you want more examples, then there are a few posts in this blog that contain them, and attending a Hardwear.io or Nullcon conference might get you a view before anyone else...

Resources

My YouTube channel. Go to the 'Playlists' and look for 'Wall Games'. There are quite a few other videos here to look at, covering topics like security, anime and music...

Hardwear.IO Virtual Con 2020 Questions Only - this is the 'Questions Only' version of the Wall Challenges from the Hardwear.IO Virtual Conference 2020 held online on the 30th of April and 1st of May 2020.

Hardwear.IO Virtual Con 2020 Questions and Answers

Hardwear.IO 2018 Questions

Hardwear.IO 2018 Questions and Answers

Hardwear.IO 2017 Questions

Hardwear.IO 2017 Questions and Answers

Hardwear.IO are excellent hardware security conferences! 

Nullcon is a recommended security conference...

The Nullcon 2020 Badge meta-puzzle...

---

If you find my writing helpful, informative or entertaining, then please consider visiting the following  link for my Synthesizerwriter alias (I write several blogs, but it makes sense to only have one 'Coffee' donation link!):

Buy me a coffee

Entropy - what is it, what does it mean and why is it useful?

Entropy is fascinating. It is a measure of the disorderliness of a system - and so, not unsurprisingly, it has a tendency to increase. You can't un-bake a cake, as they say. Organising, sorting and tidying are all counter-measures, but like recycling, it turns out that the real universe is more complex and tricky to work with than you might hope, and so trying to do 'the right thing' often has the opposite effect.

Measuring entropy is one of the activities that cryptographers use when designing random number generators. I always like the irony of a universe where there is an increasing amount of disorder, but when you want to make a source of random-ness, then it always seems to be infected with hidden layers of predictability. Random noise seems to be one of those nice ideas that is very difficult to obtain in practice, because there are a huge number of repetitive, predictable sources of information that drown out the random-ness. So random noise generation becomes a search for how to remove bias, or interference from non-random sources, followed by the refinement of statistical techniques to show how successful the removal has been.

As I have noted before, the better the encoding and encryption technique used, the more the resulting output looks like random noise. My 'think about things from a different viewpoint' brain takes that and inverts it, arriving at the conclusion that truly random noise must contain hidden information - but we just don't know the key (or the coding scheme!) and so can't decode it. I'm sure there is lots of money to be made from this type of thing, and, of course, there is!

Hackers also use entropy, but often for different reasons. If cryptographers have put lots of effort into making encryption produce noise-like data, then one way to look for encrypted data is to look for randomness. Firmware is a good place to look, because it is usually an intrinsic part of the underlying platform that services are built on, and so is an appealing target. So hackers look at firmware in a number of ways, which I always think of as different 'filters'...

The first filter that most people use is my old favourite: ASCII characters and strings. Open the extracted firmware (or data file containing it!) in a hex editor, and look for readable characters - for some reason, hex editors always seem to have a strict ordering of columns: index on the left, then the data in hex, and then the same data in ASCII printable characters. In the old 8-bit days, then you could expect to see lots of strings containing everything that appeared on the screen as text, plus lots of things that the programmer probably didn't want you to ever see: debug messages, passwords or cheat codes for reviewers/managers, initialisation settings, and more. As time has gone on, there is less and less useful information to be found, but sometimes... Wherever valuables are hidden, then there are people who will make tools to help the search. So truffleHog is just one example of a utility that searches for interesting strings on GitHub, and there are lots of other 'Information Gathering' and 'Reconnaissance' tools that will attempt to locate strings. The example screenshot above shows HexFiend,  a hex editor for MacOS, looking at itself...

What I look for in a hex editor, of course, is the ability to add offsets, to rotate, to invert and to do other 'ASCII-obscuring'-related processes in order to try and locate obfuscated strings...

Another filter that is used relates to randomness. Utilities like binwalk provide a graph that shows Entropy on the vertical axis, and index (position within the file) on the horizontal axis (binwalk does lots of other things too!). The binwalk example file shows exactly the sort of graph that the hacker probably isn't looking for: on the left, a short block with low entropy, followed by a big block with an entropy of about 0.6, followed by another short block with low entropy, then a block of about 0.5... and finally, a long block with low entropy. Nowhere in this graph is a flat block up at the 1.0 level, which would usually be inferred to mean either encrypted data, or better, keys!

(As often happens, one thing leads to another, so this thinking about binwalk took me in a very different direction, which will feature in a future blog post...)

The problem with just typing >binwalk -E filename.bin and then looking at the graph is that people get told that a flat line at 1.0 means encrypted or key data (with a short flat spike probably indicating keys!), a rough line peaking at 1.0 but with little spikes down to 0.9 or so means compressed, flat areas with zero entropy are just fixed data, and anything else is code. There's a problem here, and it relates to knowing what entropy is.

Remember that entropy is a measure of disorder. So when it is up at 1.0, it means that each successive value is very different to the previous one, and so on. - random values! Conversely, down at 0, it means that each successive value is no different to the previous one, etc. Note that 'no different' does not mean: '00 00 00 00 00...', it means lots of repeated values. So the graph doesn't show the values, it shows the variability instead.

With all of this in mind, we can now think about hiding data. If I was trying to hide keys or encrypted code inside firmware, knowing that people would use binwalk to find those keys, then all I need to do is to change the variability of the keys or encrypted code. One simple way to do this is to just add zero bytes (or any other value) every n bytes. If we add 00 every other byte then we have an entropy of zero for the 00 bytes, and 1 for the keys or code, giving an overall entropy of 0.5. We have doubled the size of the data, but for keys this does not matter, and extracting the keys is trivial! However, if you look at the obscured key data in a hex editor, then all of those 00 bytes will be obvious...

Alternatively, you could split the keys or code into nibbles: half bytes. So 7E would become 70 and 0E. If I do this to lots of data, then there are only 32 different values: 0-F followed by 0, or 0 followed by 0-F. The entropy is now not anywhere near 1.0 any longer, because all those zeroes reduce the variability - you can predict that there will be a zero either at the start or the end of the two character hex digit. But the data isn't fixed either (it isn't just 7E 7E 7E 7E 7E...), and so the entropy is about 0.5. Not only that, but looking at the data in a hex editor, it is going to be much harder to spot the zeroes. The cost of this better obfuscation is that the extraction of the keys is slightly harder... So now the key data isn't going to be obvious in either binwalk or a hex editor. In other words, you can 'design' the entropy of your data if you know what is being measured.

When I wrote this, it didn't seem that special. But now, when I go back to it, the idea that you can edit your data so that you control the entropy is very interesting indeed! It kind of takes back some of the power that randomness takes away from you...

---

Many articles these days use clickbait techniques to get your click, and then break that trust by never actually answering any of the questions they pose. So, let's prevent this right here and now:

Entropy - what is it?

The second sentence (I've never recovered fully from the Amazon review of my book where someone counted the number of pages before I actually said what something did...) tries to describe entropy without being a definition: 'a measure of the disorderliness...' . But two sentences later, my favourite is the 'un-bake a cake' metaphor. Maybe I should have said: You can't un-write a blog post...'!

Entropy - what does it mean?

This is the real fascinating thing about entropy: the more efficient you encode or encrypt something, the higher the entropy, and the more it stands out and waves a flag saying: 'Here I am!'. And the mitigation that I suggest is to reduce that efficiency by making it bigger! I suppose that having it secure and hard to find is a good compromise, but I'm now wondering if there is a way of reducing the entropy algorithmically that is hard to detect - and ultimately, the optimum way of doing that would probably also look something like noise - but special noise that has the effect of reducing the entropy of the actual data than looks like noise because it is efficiently encrypted or encoded. Unfortunately, I'm not a good enough cryptographer to to able to figure out if this can be done, but I'm always open to feedback!

Entropy - why is it useful?

The third paragraph is my 'reverse way of looking at the way the world works' moment: You can use entropy as a measure of how good your encoding or encryption is. I've always loved the way that this is the opposite of sayings like 'the perfect shape is a circle...'. Playing double jeopardy then, it follows that the more ordered a system is, the worse the encoding or encryption. In this case, the universe contains a lot of very badly encoded and encrypted information, and I am very, so very glad that we don't have the keys or the algorithms!

---

If you find my writing helpful, informative or entertaining, then please consider visiting the following  link for my Synthesizerwriter alias (I write several blogs, but it makes sense to only have one 'Coffee' donation link!):

Buy me a coffee