Review and Questions for Week 6
Professor Yuefeng Li
School of Computer Science, Queensland University of Technology (QUT)
Evaluation overview
Copyright By cscodehelp代写 加微信 cscodehelp
Evaluation is the key to making progress in building better search systems or IR models. One of the primary distinctions made in the evaluation of search systems is between effectiveness and efficiency. Effectiveness measures the ability of the search system to find the right information, and efficiency measures how quickly this is done.
The retrieval and indexing techniques have many parameters that can be adjusted to optimize performance, both in terms of effectiveness and efficiency. Typically, the best values for these parameters are determined using training data and a cost function. Training data is a sample of the real data, and the cost function is the quantity based on the data that is being maximized (or minimized).
The optimization process would use the training data to learn parameter settings for the ranking algorithm that maximized the effectiveness measure.
One of the basic requirements for evaluation is that the results from different techniques can be compared. To do this comparison fairly and to ensure that experiments are repeatable, the experimental settings and data used must be fixed.
We usually need to find a suitable data collection to test proposed models or algorithms. For example, AP (Associated Press) newswire documents from 1988–1990, which was created as part of the TREC conference series sponsored by the National Institute of Standards and Technology (NIST).
The queries for the AP are based on TREC topics. The topics were created by government information analysts employed by NIST. Queries are the title fields from TREC topics 51–150. Topics and relevance judgments generated by government information analysts.
Example of a TREC topic
Number: 794
pet therapy<br />
<desc> Description:<br />
How are pets or animals used in therapy for humans and what are the benefits?<br />
<narr> Narrative:<br />
Relevant documents must include details of how pet! or animal!assisted therapy is or has been used. Relevant details include information about pet therapy programs, descriptions of the circumstances in which pet therapy is used, the benefits of this type of therapy, the degree of success of this therapy, and any laws or regulations governing it. </top><br />
Relevance Judgments<br />
The relevance judgments depend on the task that is being evaluated. For example, TREC analysts judged a document as relevant if it contained information that could be used to help write a report on the query topic (here we primarily focused on topical relevance).<br />
Relevance judgments are normally binary, meaning that a document is either relevant or non-relevant. For some tasks, multiple levels of relevance may be appropriate, e.g., three levels of relevance (relevant, non-relevant, and neutral or unknown).<br />
Data collection can be divided into three parts (training set, validation and/or test set). We will discuss the partitions of data collection for information filtering and text classification.<br />
Creating a new test collection can be a time-consuming task since relevance judgments require a considerable investment of manual effort.<br />
Question 1. Which of the following are False? and justify your answer.<br />
(1) For a very large data collection, pooling technique is used to select top-k results from the rankings obtained by different retrieval algorithms. The results are merged into a pool, duplicates are removed, and the documents are presented in some random order to the people doing the relevance judgments.<br />
(2) Pooling produces a large number of relevance judgments for each query. However, this list is incomplete, and for a new retrieval algorithm that had not contributed documents to the original pool, this could potentially be a problem.</p>
<p>(3) Many user actions (e.g., Query log data) can be considered implicit relevance judgments. The main drawback with this data is that it is not as precise as explicit relevance judgments.<br />
(4) A typical query log does not contain user identifier or user session identifier because of the privacy issue.<br />
Question 2. (Effectiveness Metrics)<br />
Assume Table 1 illustrates the relevant judgments for documents 110 to 124 about topic 101, where 1 means the corresponding document is relevant and 0 means non-relevant; and Table 2 shows IR model1’s output, where documents are sorted according to their weights.<br />
Table 1. Relevant Judgements Table 2. IR Model1’s Output (ranked by weight)<br />
Assume A is the relevant set of documents for topic 101, A’ is the non-relevant set. We also assume IR model1 selects top-6 as the relevant documents B (i.e., the set of retrieved documents), and B’ is the set of documents that are not retrieved.<br />
(1) List all sets’ elements and enclose them in braces.<br />
(2) Calculate recall and precision of IR model1.<br />
(3) Calculate false positive and false negative of IR model1. (4) Calculate IR model1’s F1 measure.<br />
TOPIC DocNo Rel<br />
101 110 1<br />
101 111 1<br />
101 112 0<br />
101 113 1<br />
101 114 1<br />
101 115 0<br />
101 116 1<br />
101 117 1<br />
101 118 0<br />
101 119 1<br />
101 120 0<br />
101 121 0<br />
101 122 0<br />
101 123 1<br />
101 124 0<br />
TOPIC DocNo Weight<br />
101 111<br />
101 112<br />
101 113<br />
101 110<br />
101 114<br />
101 119<br />
101 115<br />
101 122<br />
101 118<br />
101 116<br />
101 123<br />
101 121<br />
101 120<br />
101 117<br />
101 124<br />
The set of retrieved documents</p>
<p>Ranking Effectiveness<br />
Most of retrieval models produce ranked output. For a fair comparison of recall and precision measures, the set of retrieved documents should be defined in terms of ranking. One possibility way is to calculate recall and precision values at every rank position p.<br />
There are several techniques have been developed to summarize the effectiveness of a ranking. The first one is simply to calculate recall-precision values at a small number of predefined rank positions. For a given query, to compare two or more rankings, the precision at the predefined rank positions needs to be calculated. This effectiveness measure is known as precision at rank p. The most common versions are precision at 10 and precision at 20 or 25.<br />
Another method of summarizing the effectiveness of a ranking is to calculate precision at fixed or standard recall levels from 0.0 to 1.0 in increments of 0.1. Each ranking is then represented using 11 numbers. This idea can be extended to calculate averaging effectiveness across queries and generating recall-precision graphs (interpolation).<br />
This method has the advantage of summarizing the effectiveness of the ranking of all relevant documents, rather than just those in the top ranks.<br />
The third method, and the most popular, is to summarize the ranking by averaging the precision values from the rank positions where a relevant document was retrieved (i.e., when recall increases).<br />
Average precision has several advantages. It is a single number that is based on the ranking of all the relevant documents, but the value depends heavily on the highly ranked relevant documents. This means it is an appropriate measure for evaluating the task of finding as many relevant documents as possible while still reflecting the intuition that the top-ranked documents are the most important.<br />
Question 3. For the give relevance judgment (Table 1) and the ranked output of IR model1 (Table 2).<br />
(1) Calculate IR model1’s precision at rank 6 (position 6) and precision at rank 10 (position 10).<br />
(2) Calculate IR model1’s average precision.</p>
<p>The aim of an averaging technique is to summarize the effectiveness of a specific ranking algorithm across multiple queries (40-50 queries are commonly recommended for evaluating IR models) rather than a single query since different queries will often have different numbers of relevant documents.<br />
Given that the average precision provides a number for each ranking, the simplest way to summarize the effectiveness of rankings from multiple queries would be to average these numbers. This effectiveness measure, mean average precision (MAP), is used in most research papers and some system evaluations.<br />
Also, an average recall-precision graph is plotted by simply joining (by using MAP) the average precision points at the standard recall levels.<br />
Question 4. Fig. 1 shows the recall and precision at positions 1, 2, … , 10 from two different queries query 1 and query 2. The precision P at any standard recall level R is defined as<br />
where S is the set of observed (R, P) points.<br />
(1) Calculate MAP of the two queries.<br />
(2) For query 1, list the elements of S in curly braces.<br />
(3) Calculate query 1’s precision P at any standard recall level R.<br />
Fig 1. Recall and precision values for rankings from two different queries<br />
(4) Assume S is represented as a list of recall-precision pairs, e.g., S = [(0.2, 1.0), (0.2, 0.5), (0.4, 0.67), (0.4, 0.5), (0.4, 0.4), (0.6, 0.5), (0.6, 0.43), (0.6, 0.38), (0.8, 0.44), (1.0, 0.5)]. Define a list comprehension to calculate precisions at all standard recall levels.</p>
<p>程序代写 <a href="https://cscodehelp.com/tag/代考/">CS代考</a> 加微信: cscodehelp QQ: 2235208643 Email: kyit630461@163.com</p>
</div><!-- .entry-content -->
<footer class="entry-meta entry-meta-footer">
<span class="cat-links cat-links-single">Posted in <a href="https://www.cscodehelp.com/category/oa%e4%bb%a3%e5%81%9a/" rel="category tag">OA代做</a>, <a href="https://www.cscodehelp.com/category/paper%e4%bb%a3%e5%86%99/" rel="category tag">paper代写</a>, <a href="https://www.cscodehelp.com/category/python%e4%bb%a3%e5%86%99/" rel="category tag">Python代写</a>, <a href="https://www.cscodehelp.com/category/software-engineering/" rel="category tag">Software Engineering</a>, <a href="https://www.cscodehelp.com/category/%e5%b9%b3%e6%97%b6%e4%bd%9c%e4%b8%9a%e4%bb%a3%e5%86%99/" rel="category tag">平时作业代写</a></span><span class="tags-links tags-links-single">Tagged <a href="https://www.cscodehelp.com/tag/oa%e4%bb%a3%e5%81%9a/" rel="tag">OA代做</a>, <a href="https://www.cscodehelp.com/tag/paper/" rel="tag">paper</a>, <a href="https://www.cscodehelp.com/tag/%e9%9d%a2%e8%af%95%e4%bb%a3%e9%9d%a2/" rel="tag">面试代面</a></span> </footer><!-- .entry-meta -->
</div><!-- .entry-data-wrapper -->
</div><!-- .post-content-wrapper -->
</article><!-- #post-## -->
</div><!-- .post-wrapper-hentry -->
<div class="entry-author">
<div class="author-avatar">
<img alt='' src='https://secure.gravatar.com/avatar/587d467738f02efc1f3ad41e6550ae64?s=80&d=mm&r=g' srcset='https://secure.gravatar.com/avatar/587d467738f02efc1f3ad41e6550ae64?s=160&d=mm&r=g 2x' class='avatar avatar-80 photo' height='80' width='80' loading='lazy' decoding='async'/> </div><!-- .author-avatar -->
<div class="author-heading">
<h2 class="author-title">
Published by <span class="author-name">admin</span> </h2>
</div><!-- .author-heading -->
<p class="author-bio">
<a class="author-link" href="https://www.cscodehelp.com/author/admin/" rel="author">
View all posts by admin </a>
</p><!-- .author-bio -->
</div><!-- .entry-auhtor -->
<nav class="navigation post-navigation" aria-label="Posts">
<h2 class="screen-reader-text">Post navigation</h2>
<div class="nav-links"><div class="nav-previous"><a href="https://www.cscodehelp.com/python%e4%bb%a3%e5%86%99/%e7%a8%8b%e5%ba%8f%e4%bb%a3%e5%86%99-mcit-593-introduction-to-computer-systems-property-of-penn-engineering-2/" rel="prev"><span class="meta-nav">Prev</span> <span class="post-title">程序代写 MCIT 593 – Introduction to Computer Systems Property of Penn Engineering 2 – cscodehelp代写</span></a></div><div class="nav-next"><a href="https://www.cscodehelp.com/python%e4%bb%a3%e5%86%99/cs%e4%bb%a3%e5%86%99-ifn647-assignment-2-requirements-weighting-35-of-the-assessment-for-ifn/" rel="next"><span class="meta-nav">Next</span> <span class="post-title">CS代写 IFN647 – Assignment 2 Requirements Weighting: 35% of the assessment for IFN – cscodehelp代写</span></a></div></div>
</nav>
<div id="comments" class="comments-area">
<div id="respond" class="comment-respond">
<h3 id="reply-title" class="comment-reply-title">Leave a Reply <small><a rel="nofollow" id="cancel-comment-reply-link" href="/python%e4%bb%a3%e5%86%99/%e4%bb%a3%e5%86%99%e4%bb%a3%e8%80%83-review-and-questions-for-week-6/#respond" style="display:none;">Cancel reply</a></small></h3><form action="https://www.cscodehelp.com/wp-comments-post.php" method="post" id="commentform" class="comment-form" novalidate><p class="comment-notes"><span id="email-notes">Your email address will not be published.</span> <span class="required-field-message">Required fields are marked <span class="required">*</span></span></p><p class="comment-form-comment"><label for="comment">Comment <span class="required">*</span></label> <textarea id="comment" name="comment" cols="45" rows="8" maxlength="65525" required></textarea></p><p class="comment-form-author"><label for="author">Name <span class="required">*</span></label> <input id="author" name="author" type="text" value="" size="30" maxlength="245" autocomplete="name" required /></p>
<p class="comment-form-email"><label for="email">Email <span class="required">*</span></label> <input id="email" name="email" type="email" value="" size="30" maxlength="100" aria-describedby="email-notes" autocomplete="email" required /></p>
<p class="comment-form-url"><label for="url">Website</label> <input id="url" name="url" type="url" value="" size="30" maxlength="200" autocomplete="url" /></p>
<p class="comment-form-cookies-consent"><input id="wp-comment-cookies-consent" name="wp-comment-cookies-consent" type="checkbox" value="yes" /> <label for="wp-comment-cookies-consent">Save my name, email, and website in this browser for the next time I comment.</label></p>
<p class="form-submit"><input name="submit" type="submit" id="submit" class="submit" value="Post Comment" /> <input type='hidden' name='comment_post_ID' value='25505' id='comment_post_ID' />
<input type='hidden' name='comment_parent' id='comment_parent' value='0' />
</p></form> </div><!-- #respond -->
</div><!-- #comments -->
</div><!-- .post-wrapper -->
</main><!-- #main -->
</div><!-- #primary -->
<div id="site-sidebar" class="sidebar-area col-16 col-sm-16 col-md-16 col-lg-5 col-xl-5 col-xxl-5 order-lg-1 order-xl-1 order-xxl-1">
<div id="secondary" class="sidebar widget-area sidebar-widget-area" role="complementary">
<aside id="text-7" class="widget widget_text"><h2 class="widget-title">联系我</h2> <div class="textwidget"><p><code></code></p>
<ul>
<li><strong>网站: </strong><a href="https://www.cscodehelp.com/">https://www.cscodehelp.com/</a></li>
<li><strong>Email</strong>: kyit630461@outlook.com</li>
<li><strong>QQ</strong>:2235208643<br />
<!-- /wp:list --> <!-- wp:image {"id":10250,"width":215,"height":401,"sizeSlug":"large"} --><!-- /wp:image --> <!-- wp:image {"id":44,"width":271,"height":307,"sizeSlug":"large"} --><!-- /wp:image --></li>
<li><strong>Wechat:</strong> aplg6666</li>
</ul>
<figure class="wp-block-image size-large is-resized"><img decoding="async" loading="lazy" class="alignnone size-medium wp-image-26377" src="https://www.cscodehelp.com/wp-content/uploads/2023/05/WechatIMG181-202x300.png" alt="" width="202" height="300" srcset="https://www.cscodehelp.com/wp-content/uploads/2023/05/WechatIMG181-202x300.png 202w, https://www.cscodehelp.com/wp-content/uploads/2023/05/WechatIMG181-690x1024.png 690w, https://www.cscodehelp.com/wp-content/uploads/2023/05/WechatIMG181-768x1140.png 768w, https://www.cscodehelp.com/wp-content/uploads/2023/05/WechatIMG181.png 802w" sizes="(max-width: 202px) 100vw, 202px" /></figure>
<p>为了保证我尽快联系和评估您的面试，作业, 请注明您的面试，作业具体要求、微信号和年级</p>
<p>Notice: 一般在美东时间 11:00 AM – 17:00之间可能回复稍慢，望谅解</p>
<p><!-- wp:paragraph {"textColor":"vivid-red","fontSize":"medium"} --></p>
<p class="has-text-color has-medium-font-size has-vivid-red-color"><strong>100% Plagiarism Free 代码保证唯一</strong></p>
<p><!-- /wp:paragraph --> <!-- wp:paragraph {"textColor":"vivid-red","fontSize":"medium"} --></p>
<p class="has-text-color has-medium-font-size has-vivid-red-color"><strong>100% Confidentiality 完全保密</strong></p>
<p><!-- /wp:paragraph --> <!-- wp:paragraph {"textColor":"vivid-red","fontSize":"medium"} --></p>
<p class="has-text-color has-medium-font-size has-vivid-red-color"><strong>100% Quality Assurance 保证质量</strong></p>
<p><!-- /wp:paragraph --></p>
</div>
</aside><aside id="text-2" class="widget widget_text"><h2 class="widget-title">请大家注意</h2> <div class="textwidget"><p>我从18年开始做代面，12年开始做代写，跌跌撞撞走来。若干网站内容都抄袭，请大家提高警惕。</p>
</div>
</aside><aside id="text-5" class="widget widget_text"><h2 class="widget-title">友情提示 </h2> <div class="textwidget"><p>我的Google排名是靠质量和口碑做起来的, 和带Ad标识的花钱排名不一样。codehelp的排名从来不需要通过付钱刷存在感</p>
</div>
</aside><aside id="categories-3" class="widget widget_categories"><h2 class="widget-title">成功案例分类</h2><form action="https://www.cscodehelp.com" method="get"><label class="screen-reader-text" for="cat">成功案例分类</label><select name='cat' id='cat' class='postform'>
<option value='-1'>Select Category</option>
<option class="level-0" value="212">AI</option>
<option class="level-0" value="214">bigdata</option>
<option class="level-0" value="1">C/C++ 代写</option>
<option class="level-0" value="175">Compilers: Principles</option>
<option class="level-0" value="253">computer architecture</option>
<option class="level-0" value="251">computer graphics</option>
<option class="level-0" value="133">computer network</option>
<option class="level-0" value="54">computer system计算机系统</option>
<option class="level-0" value="259">computer theory</option>
<option class="level-0" value="32">Data Analysis/数据分析</option>
<option class="level-0" value="33">Data Science</option>
<option class="level-0" value="208">data visualization</option>
<option class="level-0" value="236">DATABASE</option>
<option class="level-0" value="165">Distributed System 分布式系统</option>
<option class="level-0" value="246">essay</option>
<option class="level-0" value="217">genetic algorithm</option>
<option class="level-0" value="68">Go语言程序设计</option>
<option class="level-0" value="162">Hadoop</option>
<option class="level-0" value="226">Haskell</option>
<option class="level-0" value="229">hive</option>
<option class="level-0" value="147">IOS开发</option>
<option class="level-0" value="40">JAVA</option>
<option class="level-0" value="80">JavaScript</option>
<option class="level-0" value="86">JS</option>
<option class="level-0" value="75">Junit</option>
<option class="level-0" value="186">kaggle</option>
<option class="level-0" value="181">Machine Learning</option>
<option class="level-0" value="79">matlab</option>
<option class="level-0" value="119">matlab代写</option>
<option class="level-0" value="228">mobile system</option>
<option class="level-0" value="98">NLP</option>
<option class="level-0" value="273">OA代做</option>
<option class="level-0" value="39">OOP编程</option>
<option class="level-0" value="237">ORACLE SQL</option>
<option class="level-0" value="239">paper代写</option>
<option class="level-0" value="280">powcoder</option>
<option class="level-0" value="142">Prolog</option>
<option class="level-0" value="164">Python</option>
<option class="level-0" value="23">Python代写</option>
<option class="level-1" value="24"> Tensorflow代写</option>
<option class="level-0" value="203">R语言代写</option>
<option class="level-0" value="177">Scientific Computing Language</option>
<option class="level-0" value="29">Software Engineering</option>
<option class="level-0" value="163">Spark</option>
<option class="level-0" value="60">SPARK代写</option>
<option class="level-0" value="94">sql代写</option>
<option class="level-0" value="205">Statistical Analysis</option>
<option class="level-0" value="196">Stochastic Analysis in Finance</option>
<option class="level-0" value="193">Stylish</option>
<option class="level-0" value="148">Swift开发</option>
<option class="level-0" value="78">web</option>
<option class="level-0" value="85">WEBGL</option>
<option class="level-0" value="113">人工智能</option>
<option class="level-0" value="67">分布式系统</option>
<option class="level-0" value="93">加拿大代写</option>
<option class="level-0" value="128">动态规划DP dynamic programming</option>
<option class="level-0" value="106">商业分析</option>
<option class="level-0" value="192">图像处理</option>
<option class="level-0" value="107">图算法</option>
<option class="level-0" value="114">图算法graph</option>
<option class="level-0" value="198">工程</option>
<option class="level-0" value="36">平时作业代写</option>
<option class="level-0" value="188">强化学习|reinforcement learning</option>
<option class="level-0" value="61">推荐系统</option>
<option class="level-0" value="48">数据库</option>
<option class="level-0" value="59">数据挖掘</option>
<option class="level-0" value="47">数据结构｜Data Structure</option>
<option class="level-0" value="185">文本分类</option>
<option class="level-0" value="77">机器学习</option>
<option class="level-0" value="171">汇编</option>
<option class="level-0" value="219">深度学习</option>
<option class="level-0" value="69">澳洲程序设计</option>
<option class="level-0" value="123">生物信息工程</option>
<option class="level-0" value="26">科研代码代写</option>
<option class="level-0" value="159">移动通信</option>
<option class="level-0" value="127">算法algorithm</option>
<option class="level-0" value="204">统计分析</option>
<option class="level-0" value="170">编译原理</option>
<option class="level-0" value="49">网络安全</option>
<option class="level-0" value="99">自然语言处理</option>
<option class="level-0" value="254">计算机体系结构</option>
<option class="level-0" value="250">计算机图形学</option>
<option class="level-0" value="263">计算机编译原理</option>
<option class="level-0" value="132">计算机网络</option>
<option class="level-0" value="53">计算机网络代写</option>
<option class="level-0" value="28">软件工程代码代写</option>
<option class="level-0" value="120">边缘计算</option>
<option class="level-0" value="158">通信工程</option>
<option class="level-0" value="104">通信编程</option>
<option class="level-0" value="218">遗传算法</option>
<option class="level-0" value="108">量化代写</option>
<option class="level-0" value="197">随机过程</option>
<option class="level-0" value="274">面试</option>
<option class="level-0" value="272">面试代面</option>
<option class="level-0" value="279">骗子曝光</option>
</select>
</form>
<script type="text/javascript">
/* <![CDATA[ */
(function() {
var dropdown = document.getElementById( "cat" );
function onCatChange() {
if ( dropdown.options[ dropdown.selectedIndex ].value > 0 ) {
dropdown.parentNode.submit();
}
}
dropdown.onchange = onCatChange;
})();
/* ]]> */
</script>
</aside>
<aside id="recent-posts-3" class="widget widget_recent_entries">
<h2 class="widget-title">成功案例</h2>
<ul>
<li>
<a href="https://www.cscodehelp.com/c-c-%e4%bb%a3%e5%86%99/%e4%ba%9a%e9%ba%bboa-%e9%9d%a2%e7%bb%8f%e4%ba%9a%e9%a9%ac%e9%80%8a%e9%9d%a2%e7%bb%8famazon-oaoa%e4%ba%9a%e9%ba%bb%e7%bd%91%e7%94%b3amazon-%e4%bb%a3%e9%9d%a2amazon-oa%e4%bb%a3%e5%86%99/">亚麻OA 面经|亚马逊面经|AMAZON OA|OA亚麻网申|AMAZON 代面|AMAZON OA代写</a>
</li>
<li>
<a href="https://www.cscodehelp.com/c-c-%e4%bb%a3%e5%86%99/breaking-the-mold-unconventional-strategies-for-accelerating-your-career-path/">Breaking the Mold: Unconventional Strategies for Accelerating Your Career Path</a>
</li>
<li>
<a href="https://www.cscodehelp.com/c-c-%e4%bb%a3%e5%86%99/ng-%e9%9d%a2%e8%af%95-system-design%ef%bc%8c%e6%b1%97%e6%b5%81%e6%b5%83%e8%83%8c%e4%ba%86/">NG 面试 System Design，汗流浃背了</a>
</li>
</ul>
</aside><aside id="archives-6" class="widget widget_archive"><h2 class="widget-title">日期分类</h2>
<ul>
<li><a href='https://www.cscodehelp.com/2024/03/'>March 2024</a></li>
<li><a href='https://www.cscodehelp.com/2024/02/'>February 2024</a></li>
<li><a href='https://www.cscodehelp.com/2023/11/'>November 2023</a></li>
<li><a href='https://www.cscodehelp.com/2023/06/'>June 2023</a></li>
<li><a href='https://www.cscodehelp.com/2022/12/'>December 2022</a></li>
<li><a href='https://www.cscodehelp.com/2022/11/'>November 2022</a></li>
<li><a href='https://www.cscodehelp.com/2022/08/'>August 2022</a></li>
<li><a href='https://www.cscodehelp.com/2022/07/'>July 2022</a></li>
<li><a href='https://www.cscodehelp.com/2022/06/'>June 2022</a></li>
<li><a href='https://www.cscodehelp.com/2022/05/'>May 2022</a></li>
<li><a href='https://www.cscodehelp.com/2022/04/'>April 2022</a></li>
<li><a href='https://www.cscodehelp.com/2022/03/'>March 2022</a></li>
<li><a href='https://www.cscodehelp.com/2022/02/'>February 2022</a></li>
<li><a href='https://www.cscodehelp.com/2022/01/'>January 2022</a></li>
<li><a href='https://www.cscodehelp.com/2021/12/'>December 2021</a></li>
<li><a href='https://www.cscodehelp.com/2021/04/'>April 2021</a></li>
<li><a href='https://www.cscodehelp.com/2021/03/'>March 2021</a></li>
<li><a href='https://www.cscodehelp.com/2021/02/'>February 2021</a></li>
<li><a href='https://www.cscodehelp.com/2021/01/'>January 2021</a></li>
<li><a href='https://www.cscodehelp.com/2020/06/'>June 2020</a></li>
<li><a href='https://www.cscodehelp.com/2020/05/'>May 2020</a></li>
<li><a href='https://www.cscodehelp.com/2020/04/'>April 2020</a></li>
<li><a href='https://www.cscodehelp.com/2020/03/'>March 2020</a></li>
<li><a href='https://www.cscodehelp.com/2020/02/'>February 2020</a></li>
<li><a href='https://www.cscodehelp.com/2019/12/'>December 2019</a></li>
<li><a href='https://www.cscodehelp.com/2019/06/'>June 2019</a></li>
</ul>
</aside><aside id="text-3" class="widget widget_text"><h2 class="widget-title">口碑</h2> <div class="textwidget"><p>我们的<strong>Google</strong>排名是靠质量和口碑做起来的, 和带Ad标识的花钱排名不一样。codehelp的排名从来不需要通过付钱刷存在感</p>
</div>
</aside> </div><!-- .sidebar -->
</div><!-- .col-* columns of main sidebar -->
</div><!-- .row -->
</div><!-- .container -->
</div><!-- .site-content-inside -->
</div><!-- #content -->
<footer id="colophon" class="site-footer">
<div class="site-info">
<div class="site-info-inside">
<div class="container">
<div class="row">
<div class="col">
<div class="credits-wrapper">
<div class="credits credits-blog">cscodehelp™ 博士 课程作业面试辅导 CS 计算机科学 | EE 电气工程 | STATICS 统计 | FINANCE 金融 | 程序代做 | 工作代做 | 面试代面 | CS代做</div><div class="credits credits-designer">Amphibious Theme by <a href="https://templatepocket.com" title="TemplatePocket">TemplatePocket</a> <span>⋅</span> Powered by <a href="https://wordpress.org" title="WordPress">WordPress</a></div> </div><!-- .credits -->
</div><!-- .col -->
</div><!-- .row -->
</div><!-- .container -->
</div><!-- .site-info-inside -->
</div><!-- .site-info -->
</footer><!-- #colophon -->
</div><!-- #page .site-wrapper -->
<div class="overlay-effect"></div><!-- .overlay-effect -->
<script type='text/javascript' src='https://www.cscodehelp.com/wp-content/themes/amphibious/js/enquire.js?ver=2.1.6' id='enquire-js'></script>
<script type='text/javascript' src='https://www.cscodehelp.com/wp-content/themes/amphibious/js/fitvids.js?ver=1.1' id='fitvids-js'></script>
<script type='text/javascript' src='https://www.cscodehelp.com/wp-content/themes/amphibious/js/hover-intent.js?ver=r7' id='hover-intent-js'></script>
<script type='text/javascript' src='https://www.cscodehelp.com/wp-content/themes/amphibious/js/superfish.js?ver=1.7.10' id='superfish-js'></script>
<script type='text/javascript' src='https://www.cscodehelp.com/wp-includes/js/comment-reply.min.js?ver=73f13a27fe977958cf7880e6b25eac33' id='comment-reply-js'></script>
<script type='text/javascript' src='https://www.cscodehelp.com/wp-content/themes/amphibious/js/custom.js?ver=1.0' id='amphibious-custom-js'></script>
</body>
</html>