WDF*IDF: What can the SEO miracle formula really do?

WDF*IDF is an analysis method that can be used within the scope of search engine optimisation to determine keywords and terms that sustainably increase the relevance of published texts, and, therefore, the entire website. It’s a formula that multiplies the two values of Within Document Frequency (WDF) and Inverse Document Frequency (IDF). The result is the relative term frequency (also term weighting) of a document, relative to all other web documents which also contain the keyword included in the analysis. Before the WDF*IDF analysis can be run, you first need to determine the two factors mentioned.

The WDF describes how often a particular term occurs in a document compared to all other terms it contains. To increase the validity of the determined value, the formula is based on a logarithm that prevents the central term from being weighted too heavily. The term was first mentioned in 1992 in the work of Donna Harman, in which her article “Ranking Algorithms” features the term WDF as a way to give words of a particular document a weighting value useful for information science. In Website Optimisation, the WDF value has been used for some time as an alternative to the less flexible keyword density value, which merely reflects the relative abundance of a key term. The formula for determining the Within Document Frequency is:

The individual components of the equation can be explained as follows:

Therefore, the WDF value for a term “I” in the document “j” is determined by adding the frequency of the term and 1 and dividing it by the total number of words in that document. Both values use the logarithm “log2”, which gives you more meaningful results for the term than it does in determining pure keyword density or relative frequency. An example can illustrate this:

An examined term that appears 50 times in a 1,000-word document has a Within Document Frequency of 0.57. The relative frequency, in this case, is 5 percent. If you increase the frequency of the term now for optimization purposes, to say 500, you get a WDF value of 0.9 (rounded) – i.e. a value that is around 1.5 times higher than in the original text. On the other hand, if you choose the relative value (which has now risen to 50 percent) as the basis, you will see an increase 10 times the original value.

The different components of the IDF equation can be explained as follows:

Therefore, to determine the IDF value of a term "i", divide the total number of (relevant) documents contained in the result sets by the number of documents containing the term and then add the number 1. Finally, take the logarithm “log” from the result of that calculation.

Adding N_D means that the IDF formula cannot be uniformly determined. Instead, it is the result of the frequency of all meaningful words in the examined document, as well as the underlying absolute number of documents. However, when analysing web documents for SEO purposes, the potential results are huge, as all pages indexed by Google (or other search engines) are eligible. Nevertheless, to obtain a specific value, the number of search results of all relevant terms in the document is determined and added. For example, in a highly simplified document that only contains the words “Search Engine Optimisation” (17,300,00 search results, December 2017) and “Web Analytics” (2,200,000 search results, December 2017), has a N_D  value of 19,500,000.

Because Within Document Frequency represents the relevance of a term within a particular document and the Inverse Document Frequency can reflect the role of a term relative to all of the search result documents, merging both values provides deep insights into the actual term frequency and potential of the term to optimise existing text content. To this purpose, it is only necessary to multiply both values, which results in the following overall formula for the WDF*IDF analysis and help determine the most exact, usable term frequency:

In principle, it means bringing all the important components together and using them to determine the validity of terms used in webtexts. Of course, the bigger the database, the more meaningful the results are. However, to make the WDF*IDF analysis useful for search engine optimisation, it must be applied to all meaningful words within a document. This would simply be too much effort to do manually, which is why using the WDF*IDF tool is part of any serious repertoire when calculating term weighting. On the one hand, these programs (see below) help to analyse the existing textual material. On the other hand, they also provide clues as to which concepts a document lacks in order to be as unique and relevant as possible.

The advantages of a comprehensive WDF*IDF analysis are obvious: the values obtained for weighting key terms serve as perfect landmarks for writing texts so that:

• they have high relevance for search engines
• they cover topics which do not have a lot of competition
• they do not have any keyword spam
• and are as unique as possible

Anyone who is dissatisfied with his or her own website rankings or strives for improved optimisation has helpful ally by utilising WDF*IDF values. Based on analysis data, copywriters can create concrete guidelines for revising their content that aren’t just aimed at increasing the keyword density or incorporating other keywords into the text.

Although WDF*IDF provides very valuable input for website optimisation, there are a few issues that should be considered before analysing and evaluating results. For example, a fundamental problem is that a WDF*IDF analysis always includes all the textual elements of a document, whether they are headings, category/product descriptions, or captions. Differentiation of the individual components won’t take place. Even if only one paragraph is too keyword-heavy or contains too few elementary terms, the analysis method won’t provide a satisfactory answer, since the frequency weighting is always evaluated for the entire document.

Another weakness of the WDF*IDF formula is that it only gets really interesting with a high word count. For shorter passages like product descriptions, smaller blog entries or news articles, the analysis does not provide meaningful, usable results. This is why it’s often unsuitable for certain websites like online stores or news portals. For sites that rely on heavy editorial work, the drawback is that WDF*IDF analysis is difficult to incorporate into the standard workflow. Since fast response times and up-to-datedness are particularly in demand here, optimising the texts after publishing would be a practical, if complex, solution.

There are several tools that can be used to perform a WDF*IDF analysis. It is important to distinguish between applications that are only part of an SEO suite and those that are available as standalone solutions. Depending on the range of functions and the usage options, the individual tools differ in terms of cost. To give a brief overview of the variety of applications, we have compiled some of the best WDF*IDF tools in the following list:

• OnpageDoc: If you would like to analyse and optimise your websites‘ SEO status, you can use OnpageDoc, the complete package from SAC Solutions GmbH in Cologne, Germany. If you take out a monthly subscription, you’ll have access to a variety of features to review and improve keywords, meta tags, backlinks and more. A WDF*IDF tool for term weighting analysis and targeted competitive comparison is also part of the portfolio. Those who do not want to access the entire suite can also download the tool for free at wdfidf-tool.com. However, the problem is that the number of possible queries is limited to 100 queries per hour (common to all users).
• SEOlyze: Semantic analysis and research based on the WDF*IDF principle can also be done with the paid content analysis section of SEOlyze. Helminger GmbH, which is based in Austria, focuses on helping clients perfect website content and offers various tools like a W-questions tool for research, a duplicate content checker or readability analyses (Flesch/Wiener factual text formula) to achieve this. The centerpiece, however, is the comprehensive WDF*IDF analysis function, the results of which can be implemented directly into the SEOlyze interface, thanks to the integrated editor. In addition to the WDF*IDF tool, the SEO suite includes various rank-tracking features, as well as several other tools for general on-page optimisation (keyword analysis, metadata checker, images, links, etc.).
   
• XOVI: XOVI, a subsidiary of Plesk since 2017, provides its customers with a SEO suite that leaves little to be desired. The chargeable XOVI Toolbox, which is available in multiple languages, has three different models on offer (Pro, Business and Enterprise). It also includes tools to keep track of ads, traffic, keywords, backlinks and social signals. The XOVI TextOptimizer also includes a WDF*IDF text tool that not only calculates the relevance of terms used and suggests other terms based on the first ten Google search results pages, but also allows for direct editing.
   
• Seobility: Seobility offers numerous SEO tools free of charge on their homepage – such as a simple WDF*IDF tool. The web application allows users to parse the weighting of a term based on the WDF*IDF formula. In addition, the tool plays other terms (including frequency value) that match the word you are looking for. Access to the Seobility program is limited to five analyses per day per user. Users who create an account can have access to the advanced search settings in order to, for example, adjust the base of the logarithm, increase the number of considered search results or select the platform (desktop/mobile) to optimise for.